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ABSTRACT 

One of a series of educational packages designed for 
implementation either in a workshop atmosphere or through individual 
study, this Hot Topic guide presents a variety of materials designed 
to assist educators in designing and implementing classroom projects 
and activities centering on the topic of standardized and alternative 
assessment. The Hot Topic guide contains guidelines for workshop use; 
an overview of standardized and alternative assessment; and seven 
articles (from scholarly and- prof ess ional journals) and ERIC 
documents on the topic. A 19-item annotated bibliography of items in 
the ERIC database on the topic is attached. (RS) 
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In-Service Workshops and Seminars: 

Suggestions for Using this Hot Topic Guide as a 
Professional Development Tool 



* 



Before the Workshop: 

• Carefully review the materials presented in this Hot Topic Guide. Think about how these 
concepts and projects might be applied to your particular school or district. 

• As particular concepts begin to stand out in your mind as being important, use the 
Bibliography section (found at the end of the packet) to seek out additional resources 
dealing specifically with those concepts. 

• Look over the names of the teachers and researchers who wrote the packet articles 
and/or are listed in the Bibliography. Are any of the names familiar to you? Do any of 
them work in your geographical area? Do you have colleagues or acquaintances who 
are engaged in similar research and/or teaching? Perhaps you could enlist their help and 
expertise as you plan your workshop or seminar. 

• As you begin to plan your activities, develop a mental "movie" of what you'd like to see 
happening in the classroom as a result of this in-service workshop or seminar. Keep this 
vision in mind as a guide to your planning. 

During the Workshop: 

• Provide your participants with a solid grasp of the important concepts that you have 
acquired from your reading, but don't load them down with excessive detail, such as 
lots of hard-to-remember names, dates or statistics. You may wish to use the 
Overview/Lecture section of this packet as a guide for your introductory remarks about 
the topic. 

• Try modeling the concepts and teaching strategies related to the topic by "teaching" a 
minilesson for your group. 

• Remember, if your teachers and colleagues ask you challenging or difficult questions 
about the topic, that they are not trying to discredit you or your ideas. Rather, they are 
trying to prepare themselves for situations that might arise as they implement these 
ideas in their own classrooms. 

• If any of the participants are already using some of these ideas in their own teaching, 
encourage them to share their experiences. 

• Even though your workshop participants are adults, many of the classroom management 
principles that you use every day with your students still apply. Workshop participants, 
admittedly, have a longer attention span and can sit still longer than your second- 
graders; but not that much longer. Don't have a workshop that is just a "sit down, shut 
up, and listen session. Vary the kinds of presentations and activities you provide in 
your workshops. For instance, try to include at least one hands-on activity so that the 
participants will begin to get a feel for how they might apply the concepts that you are 
discussing in your workshop. 

• Try to include time in the workshop for the participants to work in small groups. This 
time may be a good opportunity for them to formulate plans for how they might use the 
concepts just discussed in their own classrooms. 

• Encourage teachers to go "a step further" with what they have learned in the workshop. 
Provide additional resources for them to continue their research into the topics 
discussed, such as books, journal articles. Hot Topic Guides, teaching materials, and 
local experts. Alert them to future workshops/conferences on related topics. 
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After the Workshop: 

• Follow up on the work you have done. Have your workshop attendees fill out an End- 
of-Session Evaluation (a sample is included on the next page). Emphasize that their 
responses are anonymous. The participants' answers to these questions can be very 
helpful in planning your next workshop. After a reasonable amount of time Isay a few 
months or a semester), contact your workshop attendees and inquire about how they 
have used, or haven’t used, the workshop concepts in their teaching. Have any 
surprising results come up? Are there any unforeseen problems? 

When teachers are trying the new techniques, suggest that they invite you to observe 
their classes. As you discover success stories among teachers from your workshop, 

share them with the other attendees, particularly those who seem reluctant to give the 
ideas a try. 

• Find out what other topics your participants would like to see covered in future 
workshops and seminars. There are nearly sixty Hot Topic Guides, and more are always 
being developed. Whatever your focus, there is probably a Hot Topic Guide that can 
help. An order form follows the table of contents in this packet. 



Are You Looking for Uniyersity Course Credit? 

Indiana University's Distance Education program 
is offering new one-credit-hour Language Arts Education 
minicourses on these topics: 



Elementary: 

Language Learning and Development 
Varied Writing Strategies 
Parents and the Reading Process 
Exploring Creative Writing with 
Elementary Students 

Secondary: 

Varied Writing Strategies 
Thematic Units and Literature 
Exploring Creative Writing with 
Secondary Students 

K-12: 

Reading across the Curriculum 
Writing across the Curriculum 
Organization of the Classroom 

Course Requirements: 

These minicourses are taught by 
correspondence. Minicourse reading 
materials consist of Hot Topic Guides and 
ERIC/EDINFO Press books. You will be 
asked to write Goal Statements and 
Reaction Papers for each of the assigned 
reading materials, and a final Synthesis 
paper. 



/ really enjoyed working at my own pace.... 
it was wonderful to have everything so 
organized.. .and taken care of in a manner 
where / really felt like / was a student, 
however "distant" / was.... " 

--Distance Education student 

Three-Credit-Hour Courses 

are also offered (now ^th optional 

videos!): 

Advanced Study in the Teaching of: 

• Reading in the Bementary School 

• Language Arts in the Bementary School 

• Secondary School English/Language Arts 

• Reading in the Secondary School 
Writing as a Response to Reading 
Developing Parent Involvement Programs 
Critical Thinking across the Curriculum 
Organization and Administration of a 

School Reading Program 

For More Information: 

For course .outlines and registration 
instructions, please contact: 

Distance Education Office 
Smith Research Center, Suite 1 50 
2805 East 1 0th Street 
Bloomington, IN 47408-2698 
1-800-759-4723 or (812) 855-5847 



Planning a Workshop Presentation 

Worksheet 

Major concepts you want to stress in this presentation: 

1 ) 

2 ) 

3 ) 

Are there additional resources mentioned in the Bibliography that would be worth 
locating? Which ones? How could you get them most easily? 



Are there resource people available in your area whom you might consult about this 
topic and/or invite to participate? Who are they? 



What would you like to see happen in participants’ classrooms as a result of this 
workshop? Be as specific as possible. 



Plans for followup to this workshop: [peer observations, sharing experiences, etc.] 



V 



Agenda for Workshop 

Planning Sheet 



Introduction/Overview: 

[What would be the most effective way to present the major concepts 
that you wish to convey?] 



Activities that involve participants and incorporate the main concepts of this workshop: 

1 ) 



2 ) 



Applications: 

Encourage participants to plan a mini-lesson for their educational setting that 
draws on these concepts. [One possibility is to work in small groups, during 
the workshop, to make a plan and then share it with other participants.] 

Your plan to make this happen: 



Evaluation: 

[Use the form on the next page, or one you design, to get feedback from 
participants about your presentation.] 




EnD-OF-SESSION EVflUlflTION 



Now that toda 3 ^s meeting is over, we would like to know how you feel and what you think about 
the things we did so that we can make them better. Yoxu* opinion is important to us. Please 
answer all questions honestly. Your answers are confidential. 



1. Check ( »/ ) to show if todays meeting was 

Q Not worthwhile Q Somewhat worthwhile Q Very worthwhile 

2. Check ( t/ ) to show if today’s meeting was 

□ Not interesting □ Somewhat interesting Q Very interesting 

3. Check ( ^ ) to show if toda 3 ^s leader was 

□ Not very good Q Just O.K. □ Very good 

4. Check ( »/ ) to show if the meeting helped you get any useful ideas about how you 
can make positive changes in the classroom. 

□ Very little Q Some □ Very much 

5. Check ( ) to show if today’s meeting was 

□ 11)0 long Q Ibo short □ Just about right 

6. Check ( »/ ) whether you would recommend today’s meeting to a colleague. 

□ Yes GNo 

7 . Check ( «/ ) to show how useful you foimd each of the things we did or discussed today. 
Getting information/new ideas. 

□ Not useful □ Somewhat useful □ Very useful 

Seeing and hearing demonstrations of teaching techniques. 

□ Not useful □ Somewhat useful □ Very useful 

Getting materials to read. 

□ Not useful □ Somewhat useful Q Very useful 



Listening to other teachers tell about their own experiences. 

□ Not useful □ Somewhat useful Q Very useful 

Working with colleagues in a small group to develop strategies of our own. 

□ Not useful □ Somewhat useful Q Very useful 

Getting support from others in the group. 

□ Not useful □ Somewhat useful Q Very useful 

8. Please write one thing that you thought was best about today: 



9. Please write one thing that could have been improved today: 



10. What additional information would you have hked? 



11. Do you have any questions you would like to ask? 



12. What additional comments would you like to make? 



Thank yoa for completing this form. 



STANDARDIZED AND ALTERNATIVE ASSESSMENT 

Overview by 

Mardziah Hayati Abdullah 
M.S. Language Education, Indiana University 



We evaluate every day. We make judgments about the most common things 
without even realizing it when we comment on how a dish tastes, which TV program 
is more entertaining or how to perform a chore better. Understandably, we would 
expect evaluation to be an important part of a system in which so much national 
interest and expenditure is invested: education. Evaluation should inform educators 
about how well they have done or are doing, and it should indicate how to educate 
better. However, educational assessment has other social and political consequences 
which significantly impact society and thus provoke debate. This overview introduces 
some of the major issues surrounding educational assessment and provides a guide to 
further reading on particular aspects. 

Trends in Assessment 

The practice of evaluating human ability and performance in an organized 
manner has been around for centuries, dating as far back as 2000 B.C. when Chinese 
civil examinations were established in a move toward appointing civil servants based 
on meritocracy instead of unfair preference. Since then, trends in assessment have 
undergone changes in response to changes in the philosophy of education, 
assumptions about learning, and political ideologies. 

In the 400's B.C, the Greek philosopher Socrates used oral conversational 
methods for examining rhetorical abilities in presenting and defending arguments. 
Much later, in the 1700's, early testing in the United States also involved oral 
examinations conducted by faculty who used their expert judgment to determine the 
quality of their students' performances, much like the oral defenses required of 
doctoral students today. Horace Mann started using written standardized tests in 
Massachusetts and Connecticut in the 1830's, leading to the development of Mann's 
Boston Survey in 1846, the first printed test for large-scale assessment of student 
achievement in various disciplines. The tests were discontinued because the results 
were not used. 

Standardized examinations were again recommended after 1895 when Joseph 
Rice conducted tests in a number of large school systems that yielded large 
differences in math scores among schools. In the 1900's Thorndike, often called the 
father of the educational testing movement, persuaded educators to measure human 
change. There was a call for immediate, demonstrable results, and by 1915, testing 
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had become the primary means of evaluating schools. A lack of trust in teachers 
developed because of the assumption that they were biased in evaluating students, 
and standardized norm-referenced tests emerged so that individual students' or group 
scores could be compared against those of other individuals or groups on a 
bell-shaped normal curve. Dewey's progressive education movement favored 
education based on natural learning and practical problems in the 1930's, and while 
that may have called for more authentic testing, standardized norm-referenced testing 
was still employed, leading to a boom in the technical development of tests to 
compare schools on a larger scale. 

As educators realized that standardized test outcomes did not adequately 
reflect the complexity of learning, opposition against standardized testing grew. The 
1980's saw a rise in the use of criterion-referenced tests that, in reality, are also 
standardized, but students are assessed against a set of criteria instead of other 
students. By the 1990's, however, educators were raising and documenting more and 
more problems with standardized testing. In answer, educators explored alternatives 
to standardized testing which fall under the umbrella term alternative assessment. 
Different forms of alternative assessment are still being developed and tried 
enthusiastically by some educators, but they, too, face opposition. In Chapter I and 
part of Chapter 8 in the book Toward a New Science of Rdiicatinnal Testing and 
As^cgsment, which have been included in this Hot Topic guide, Berlak discusses 
some of the epistemological contentions in the history of evaluation. 

Testing and assessment 

Before engaging in a discussion on standardized testing and alternative 
assessment, I think it is important to read Chapter 1 in the reading text for this 
guide, Assessing Student Performance: Exploring the Purpose and l imits of Testing 
in which Wiggins (1993) makes an enlightening distinction between testing and 
assessment. 

In a nutshell, a test is a "one- shot" procedure which assumes to measure the 
test-taker's ability or knowledge. It requires the test-taker to provide uniform, 

'correct' responses to items formulated and scored by other parties using criteria over 
which the test-taker has no say. Thus, Wiggins claims, to "test" a student is "a 
practice of determining whether the student has mastered what is orthodox" (p. 10). 
Tests are often "secure" (the contents kept secret) before they are administered. The 
role of human judgment in scoring is deliberately minimized. Standardized items and 
scoring also necessarily minimizes responsiveness to individual test-takers and 
contexts. Assessment, in contrast, is a comprehensive, multi-faceted analysis of 
performance; it must be judgment-based and personal" (p.l3). By Wiggins' definition, 
assessment requires the systematic collection of data about a student's performance 
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using multiple and varied techniques, including assessors' observations. Unlike scores 
obtained from one-shot tests, assessment involves the "integration of (diverse) 
information in a summary judgment" about a student (p.l3) and necessitates the 
assessor's personal and subjective evaluation of a particular individual's performance 
in a particular context. The student has more say in his own assessment. 

If one accepts the distinctions described above, then testing cannot be 
assessment, and vice-versa. Wiggins' book clarifies aspects of evaluation that further 
distinguish the two, such as the practice of assessment (Chapter 2), test security 
versus tact (Chapter 4) and the nature of feedback (Chapter 6). (Note: Chapter 6 is 
not a required reading for this Hot Topic guide but is included in the one on 
Measurement Issues). 

Standardized testing 

A standardized test is typically one in which the same questions, in the same 
form, are given to everyone who is being evaluated. The test may be scored by a 
machine or by different raters following a common rubric. On one hand, standardized 
testing procedures have made evaluation convenient by offering neatly packaged and 
often easily obtainable test items. Machine scoring of multiple- choice questions has 
also made scoring easier and free of human error, and even Wiggins acknowledges 
that there are virtues in minimizing bias and error in human judgment. However, 
more and more educators are presenting strong arguments against standardized 
testing. For instance, bias can also be built into the questions themselves, as test 
items inevitably reflect particular perspectives that may not be shared by test-takers. 
Thus, incorrect responses need not indicate erroneous understanding. 

Berlak discusses four assumptions underlying what he considers to be the 
current psychometric paradigm of standardized testing, along with counter-arguments 
that support the emergence of a contextual or curriculum-sensitive paradigm. One 
assumption is that there can be a single, established consensus about what a test 
score means about individuals all over the world, while the argument is that there are 
"plural and contradictory" perspectives on what it means to be able or competent in a 
certain area. Secondly, the psychometric paradigm assumes that ' scientific' 
techniques and instruments are value-neutral and thus should yield objective 
measurements; this is countered by the assertion that there are different learning 
situations and culturally ingrained perspectives, making it morally wrong to measure 
students' performance without considering those factors. Another assumption is that 
cognitive learning can be separated from affective learning and measured likewise, but 
the contextual paradigm argues that cognitive ability cannot be assessed 
independently from other human factors. Finally, the psychometric paradigm sees a 




need for centralized control, whereas assessment reform calls for decentralized 
decision making. 

Standardized testing has also raised important issues of equity, 
deprofessionalization of teachers and the negative use of test scores for purposes such 
as placement, tracking and grade retention. Pages 5-18 of Linda Darling-Hammond's 
article Symposium: Equity in Educational Assessment (included in this guide) 
presents these issues convincingly, highlighting the need to consider the 
consequential validity of testing. Haladyna, Nolen and Haas, in their 1991 article. 
Raising Standardized Achievement Scores and the Origins of Test Score Pollution 
(found in this guide), point out yet another consequence: unethical practices that 
schools engage in as a result of society's concern with standardized scores as 
indicators of performance. The escalating academic demand has even led to readiness 
screening and grade retention being imposed on children as young as kindergartners 
(Shepard & Smith, 1988). With these and other criticisms being levied against 
standardized testing, assessment, as Wiggins defines it, is increasingly being sought as 
an alternative evaluation approach. 

Alternative assessment 

Alternative assessment has come to be used as an umbrella term that covers 
authentic assessment and performance assessment, and incorporates techniques such 
as portfolio assessment, exhibitions of mastery, projects, profiles and discourse 
assessment. 

Authentic assessment refers to the assessment of achievement or ability shown 
through performances which are modeled on real-life tasks or practices in 
beyond-school settings. It is performance -based as it evaluates performances such as 
the ability to integrate, apply or produce knowledge in contexts representative of or 
similar to real-life situations that (ideally) have personal value for the student. The 
forms and criteria of assessment are similar to those that might be used in real 
situations. Thus, authentic assessment is likely to have a combination of some or all 
of the following characteristics: ongoing, continuous, personalized, negotiable, with multiple 
indicators, and possibly conducted by more than one assessor, although this list is not 
exhaustive. Fred Newmann and Douglas Archbald discuss criteria for authentic 
assessment in Chapter 4 of the Berlak (et.al.) book (not included in the guide). In 
Chapter 7 (also not included), the same writers describe practices for assessing 
academic achievement that meet one or more of those criteria. 

Wiggins' book offers one of the best discussions available on what should 
constitute performance assessment. Gitomer (in press) defines a performance task as "one 
that simultaneously requires the use of knowledge, skills and values that are 
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recognized as important in a domain of study and is qualitatively consistent with 
tasks that members of discipline-based communities might conceivably engage in." 
The alternative assessment techniques mentioned earlier represent performance tasks 
that vary in scope of content, method of initiation and method of presentation. Texts 
by the following (not required reading but listed under references at the end of this 
overview) provide more detailed descriptions of these techniques: Newmann in 
Berlak, et.al. (discourse assessment), Mabry (demonstrations of mastery), Stenmark 
(performance tasks). Archbald and Newmann (portfolios and profiles). (The last text 
is a required reading for the Hot Topic guide on Measurement Issues.) 

How do alternative assessment techniques differ from standardized tests in 
language education? Discourse assessment, as described by Newmann, is an example: 
it is the assessment of a student's ability to present coherent, 'whole' (as opposed to 
fragmented) responses in discursive forms such as narratives or arguments, which 
reflect synthesized, personal, critical and contextually appropriate use of content and 
language. It also helps to understand what discourse assessment is not, namely, the 
checking of answers to responses, both of which are presented in a sequence that has 
no obvious meaning or purpose beyond that of testing discrete items of knowledge. 
Discourse assessment is in principle authentic assessment, as it requires a form of 
discursive performance that we engage in real-life situations, such as when we write 
letters, argue to defend our position or choices, negotiate with others or present 
information. However, authenticity comes only with contextual appropriateness, 
hence we must consider the nature of the performance task and criteria. For instance, 
if students in a rural village (who are likely to become the future leaders) were facing 
the possibility of their only soccer field being used for the construction of teachers' 
quarters, an authentic assessment task would be for them to present strong oral or 
written arguments for stopping the housing project. The assessor must also be familiar 
with the discursive elements that are culturally appropriate for that situation, 
otherwise an inauthentic assessment will be rendered. A standardized test necessarily 
excludes such personalized attention. 

Indeed, Wiggins and Mabry view alternative assessment within the framework 
of a personalized assessment paradigm. Going beyond aligning curriculum with 
assessment, which is the crux of a contextual paradigm, a personalized paradigm 
includes student- sensitive content, variable times and settings, greater student 
selection and the essential feature of self-assessment by students in addition to 
evaluation by others. The intent is to find out about a particular student in a 
particular setting over a period of time, and giving him the opportunity to show what 
he does best in the way he does it best. Undoubtedly, there are numerous difficulties 
associated with such a paradigm: among them are that teachers and students need far 
more power than they have at the moment and they need to be very clear about the 
nature of performance assessment, schools, curricula and schedules have to be 
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restructured to accommodate more flexible assessment practices, and society has to 
stop evaluating schools in terms of statistical scores. Despite the problems, however, 
personalized alternative assessment may at the moment be the only way to counter 
the absence of student-centeredness in standardized testing. 

Rubrics and scoring 

Assessment reform involves more than merely changing the nature of the tasks. 
A lot of thought has to be given to what kind of corresponding change there should 
be in the way performance tasks are scored. For example, some teachers vehemently 
oppose giving grades, so should grades be used at all? If they are not, how does one 
document student achievement? Keeping descriptive records of students' progress and 
performance is one option. Portfolios of students' work is another. But if grades have 
to be given, what are the standards by which they are determined? There is no easy 
answer. For example, although the Vermont portfolio assessment program was an 
attempt to move away from standardized testing, their writing assessment still used 
an assessment rubric that required every student to meet the same specified criteria 
broken down analytically into dimensions such as purpose, details, organization and 
mechanics. Essentially, the students were still being evaluated in a standardized 
manner. 

Test security 

The perceived need for test security is another major difference between 
standardized testing and alternative assessment. The assumption that there are 
certain 'correct' answers necessitates keeping the content of tests confidential, while 
assessment calls for open negotiation between assessorand student. 

One reasonable argument for supporting test security is Wiggins': open 
knowledge of test content may cause students to focus only on limited domains of 
learning. However, open performance-based assessment can be designed to ensure the 
need to integrate knowledge from various domains. Test situations that I can think of 
which may warrant security are ones which need to assess the appropriateness of 
spontaneous response, such as spontaneous medical decisions; in such cases, I feel it 
is justifiable to keep both test content and format unknown to the student before 
the test, but discussion on results must be open. 

There are, however, many reasons why tests should not be secure. The first 
concern is with authenticity. Many forms of assessment in the real world beyond 
educational settings do not practice the kind of secrecy adhered to in school settings, 
particularly in standardized testing; thus, going through secure tests is not an 
authentic assessment experience. Security generally requires one-shot testing in 



artificially constructed settings (for example, secure rooms); subsequently, the 
validity of interpretations made of students' achievement in secure tests can be 
questioned, since it is not an assessment of authentic use of knowledge. 

Secondly, although test security is intended to ensure credibility and fairness 
in test administration, the irony is that secrecy actually leads us to challenge those 
very aspects. Secrecy cuts off dialogue between student and assessor and makes it 
impossible for students (as well as teachers and parents, in the case of state-wide 
tests) to raise objections or ask for clarification before tests are administered. 
Inevitably, this leads to doubts about the credibility of the tests, although objections 
are rarely entertained. In some situations, hours- long quarantine may be imposed on 
students before an exam in which there is suspected leakage of test information. Both 
the validity and reliability of student performance under such conditions can be 
seriously called into question; more often than not, however, no one does so, 
particularly not the powerless students, even though they are the real victims. Thus, 
the moral dimension of secrecy in testing is also a subtantial cause for concern. In 
addition, to students who are denied knowledge of the standards and criteria by 
which they are judged, tests are both arbitrary and unfair. Secrecy in high-stake 
testing also leads to high anxiety, which in turn often results in unethical practices, 
such as selling and buying 'leaked' test questions, and cheating during exams. 

Thirdly, there can be very little or no instructional value when results are kept 
secret even after the test. Students are also denied consultation with or guidance from 
adults during assessment, nor do they have access to resource materials, restrictions 
that do not often exist in the real world. In addition, because the whole testing 
procedure is shrouded in secrecy and only selected personnel are allowed access to 
test planning and design (in standardized tests), test security leads to an over-reliance 
on and excessive belief in 'expert' assessment authorities and a distrust of teacher 
competence. Thus teachers have little control over testing imposed by strangers on 
their students, whom they know best. 

Reliability and validity 

The distinctions between standardized testing and alternative assessment are 
tied in to issues of reliability and validity. Both Wiggins' and Berlak's texts for this 
topic address these constructs to some extent. Reliability refers to the consistency or 
replicability of scores or performances, while validity, as Messick defines it, is the 
degree to which empirical evidence and theoretical rationales support the adequacy 
and appropriateness of inferences and actions based on test scores or other modes of 
assessment (Messick, 1989). Both standardized testing and alternative assessment 
attempt to achieve a degree of reliability and validity. Standardized tests, however, 
have a greater concern with reliability and are "intrinsically prone to sacrifice validity" 



(Wiggins, p.4); they are usually simplified for the sake of precision in scoring, and 
thus lose the authentic complexity of real-life tasks. In addition, the test security 
demands test-taking under artificial circumstances, as has been previously mentioned, 
rendering the validity of the inferences one can draw from test scores produced under 
such circumstances questionable. Alternative assessment, on the other hand, by 
focusing on more authentic ill-structured tasks, considering individual students to a 
greater degree, and allowing negotiation between the student and teacher, reduces the 
degree of reliability but raises the degree of validity of the inferences that can be 
made about the student's performance. The Hot Topic guide on Measurement Issues 
discusses these measurement constructs in greater depth. 

Conclusion 

Like most other problems that impact society in a significant way, the issues 
surrounding educational testing and assessment are not easily resolved because they 
are tied to ideology, politics and social systems. Standardized testing is familiar and 
comfortable, while numerous obstacles still have to be overcome in order to 
successfully reform assessment. Standardized test results are the basis for 
reproducing a stratified society, but true assessment reform, particularly assessment 
without traditional grades, may lead to the restructuring of society. The question is 
whether society is ready to move into a new paradigm. 
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Raising Standardized 
Achievement Test Scores and 
the Origins of Test Score 

Pollution 



Thomas M. Haladyna Susan bobbit Nolen Nancy S. Haas 



plan instruction, diagnose achievement deficits, place 
students into special programs (e.g., gifted, handicapped), 
and help parents understand the general achievement levels 
of their children. The considerable increase in the use of these 
test scores might be attnbuted to the onset of the "age of ac- 
countability and an increased perceived need to evcduate 
education at virtually all units of an<dysis (i.e., individu<ds, 
dasses, schools, school districts, states, and even the nation)! 

With the increased use of standardized achievement tests 
has come pressure to raise scores, which in turn leads to in- 
creased test score pollution. This pollution seriously affects 
the truthfulness of test score interpretations and ralk into 
doubt the reaonableness of many of the uses listed in Table 1 

In this paper we will (a) defee test score pollution, (b) 
des^ibe the nature and extent of two major sources of test 
score pollution (student preparation and test administration 
practices), (c) discuss factors leading to polluting practices, 
and (d) suggest ways to combat test score pollution. But first, 
validity of test score use should be discussed as a context for 
the problem of test score pollution as it exists today. 

The Validity Context 

To validate a particular test use or interpretation, one should Tri^ 

pn^e evidence supporting the truthfulness of that use. 

This fundamental principle is stated in standard 1.1 in the 
Stamlards for Educational Testing (1985, p. 13) 



In the current climate of dissatisfaction with public education, the 
standardized achievement test score has been the operational d^ni- 
tion for educational achievement, and raising test scores has been 
equated with educational improvement. The pressure to raise test 
has resulted in practices which pollute the inferences we make 
from these scores. We examine two major sources of test score pollu- 
tion: (a) how public school personnel prepare students to take the 
standardized test and (b) nonstandard practices and conditions under 
which tests are administered. We also examine the apparent causes 
of this pollution and its effects on testing practices in American 
education. 

Educational Raearcher, Vol. 20, No. 5, pp. 2~7 



T he coin of the realm in public education in the United 
States is the standardized achievement test score. This 
score is universally and uncritically accepted by the 
public and many educators as a valid measure of educational 
accomplishment (Haertel & Calfee, 1983). In a review of 
testing practices related to standardized achievement testing, 
Halad 3 fTna, Haas, and Nolen (1989) listed 29 possible uses of 
standardized tests (see Table 1). These uses range from p<^ky 
analysis at the national level to parental review dieirduld's 
achievement. For instance, test scores are used to rank states 
by the United States Department of Education in its aiuuial 
Report Card,' and by legislators and other government of- 
ficials to assess educational effectivOTess of states and schod 
districts. School boards and schod district persoimel use test 
scores to determine the effectiveness of their districts and 
schools within each district. Newspapers rank schocA districts 
by test scores to bemoan the failure of education. Test scores 
are used by some school district personnd to determine merit 
pay and to make other persoimel decisions. Real estate 
agents use test scores to rate neighborhoods in terms of the 
"quality of schools." 

Until recently, these test scores were used for a rather 
limited set of purposes. Scores were used to group students 
for instruction, evaluate and modify school district curricula. 
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Table 1 

Consumer and Uses of Standardized Test Information 



Consumer 


Unit of analysis 


National level 

Allocation of resources to programs and priorities 
Federal program evaluation (e.g.. Chapter 1) 


Nation, state 
State, program 


State legislature/state department of education 
Evaluate state's status and progress relevant to standards 
State program evaluation 
Allocation of resources 


State 

State, program 
District, school 


Public (lay persons, press, school board members, parents) 
Evaluate state's status and progress relevant to standards 
Diagnose achievement deficits 
Develop expectations for future success in school 


District 

Individual, school 
Individual 


School districts— central administrators 
Evaluate districts 
Evaluate schools 
Evaluate teachers 
Evaluate curriculum 
Evaluate instructional programs 

Determine areas for revision of curriculum and instruction 


District 

Schools 

Classroom 

District 

Program 

District 


School districts — building administrators 
Evaluate school 
Evaluate teacher 

Grouping students for instruction 
Placement into special programs 


School 

Classroom 

Individual 

Individual 


School districts— teachers 
Grouping students for instruction 
Evaluating and planning the curriculum 
Evaluating and planning instruction 
Evaluating teaching 
Diagnosing achievement deficits 
Promotion and graduation 

Placement into special programs (e.g., gifted, handicapped) 


Individual 

Classroom 

Classroom 

Classroom 

Classroom, individual 

Individual 

Individual 


Educational laboratories, centers, universities 
Policy analysis 
Evaluation studies 
Other applied research 

Basic research 


All units 
All units 
All units 
All units 


and more completely discussed in chapters on validity in the 
second and third editiorw of Educational Measurement (Cron- 
bach, 1971; Messick. 1989). For the present purposes, the 
validity of any standardized achievement test score use is 
conceptualized in terms of a unified approaA which em- 
bodies both content and construct considerations ^esskk. 
1989). In this validity context, any standardized achieven^t 
test score represents a generalized measure of accomplish- 
ment of school curricula (Waldrop et aL, 1982). As Me^m 
and Kaminski (1989) observed, the makers of standardize 
achievement tests hold that their tests are merely a ^ple 
from a broad achievement domain of knowled^ and skills, 
a view consistent with the conception of validity we are 
using. 

Cronbach addressed this issue and its relationdup to test 
preparation; VVhenever it iscritkally important to m^ cer- 
tain content, the knowledge that it will be testM proaui^ 
a desirable concentration of effort. On the other handr 


ing the answer to a set of questions is by no means the same 

as acquiring understanding of whatever topic that question 
represents (Cronbach, 1963, p. 681) 

No sin^e standardized adrievement test represents a com- 
plete mapping of the content of the school achievement do- 
main. nor is it so intended by its publishers. Indeed, many 
critics of standardized testing seek test use reform through 
the use of multiple indicators that better represent the com- 
plexity of school achievement. The December 1989 issue of 
the Eduattional Researcher is devoted to this idea of expanding 
the scope of educational achievement measurement. 

r fifg the achievement domain, the causes of school 
achievement are varied and complex, a fact that further corn- 
plkates the interpiretation of jKiuevement test scores. In 
Walberg's productivity model (1980) of school learning, for 
example, school achievement is a function of a variety of fac- 
tors, only some of which are under the mfluence of schools. 
Schoob can influence the quality and quantity of instruction. 
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rriotivition, and the learning environment, but they have lit- 
tle or no effect on family and home environment, maturity, 
and mental ability. Although researchers may disagree as to 
the relative degrees of influence of these factors or to the ex- 
tent they interact, there is little disagreement that all are 
important. 

Unfortunately, the consumers of school achievement test 
scores have often used test results without considering the 
complexity of achievement and its causes. Waldrop et al. 
(1982) maintain that standardized tests typically lack ^e "in- 
ference systems" necessary for many intended uses. For in- 
stance, erroneously attributing the level of achievement test 
scores to the influence of a single teacher, school, or school 
district grossly oversimplifies the nature of these scores. This 
misrepresentation seems to contribute to the growing fear 
and loathing that teachers and administrators feel toward 
standardized testing (Haas, Haladyna, & Nolen, 1989; 
Nolen, Haladyna, & Haas, 1989; Smith, with others, 1990). 

There have been an increasing number of critics of this in- 
appropriate use of test scores (HaerteL 1986; Madaus, 1988; 
Shepard, 1989). We believe that this increasing pressure to 
produce high test scores, combined with a lack of under- 
standing of the complexities of achievement and its causes, 
has led to widespread practices that we have termed test score 
pollution. 

Test Score Pollution 

Test score pollution, a concept based on the work of Messick 
(1984), refers to factors affecting the truthfulness of a test 
score interpretaton. Specifically, pollution increases or 
decrea^ test performance with^t connection to the cortstruct 
represented by the test, producing construct-ixielevant test 
score variance. Interpretations of these biased scores in- 
fluence public opinion and policy and thus affect American 
education at all levels. There is reason to believe that the 
problem of test score pollution is pervasive in American 
education. 

Three main sources of test score pollution are (a) the way 
schools and its personnel prepare students for tests, (b) test 
administration activities or conditions, and (c) exogenous fac- 
tors representing forces beyond the control of schools and 
school personnel. Although this third factor is not part of the 
present discussion, we mention it briefly here. There are 
many factors believed to be causative agents of educational 
achievement. The reporting of test scores without 
acknowledging the influence of family, family mobility, 
economic environment, proficiency with the English 
language, and other such factors can lead both lay persons 
and educators to draw invalid inferences from these test 
scores. In this article however, we shall restrict the discus- 
sion of pollution to two factors under the control of school 
|:>ersonnel. 

Polluting Practices 

Researchers have documented a number of activities aimed 
at preparing students for tests (Haladyna et al., 1989; 
Mehrens & Kanunski, 1989). These practices include (a) 
teaching test-taking skills, (b) promoting student motivation 
for the test, (c) developing a curriculum to match the test, 
(d) preparing teaching objectives to match the test, (e) pre- 
senting items sirrular to those presented on the test, (0 using 
commercial materials specifically designed to improve test 



performance, and (g) presenting before the test the actual 
Hems to be tested. 

Polluting practices also occur during the actual administra- 
tion of the tests. These include (a) "cleaning" answer sheets 
by darkening responses and erasing stray marks, (b) dismis- 
sing low-achieving students on test days, and (c) interfering 
with responses (e.g., giving hints or answers to students or 
altering response sheets). Other conditions exist in and 
around the testing situation that are known to influence test 
scores. These include students' anxiety, stress, fatigue, and 
motivation, as well as the pace of performance on the test 
as a function of the time available. This latter factor is also 
known to interact with arudety and to yield predictably poor 
results (Matarazzo, 1972). 

Table 2 provides our appraisal of the ethics of these prac- 
tices. It must be emphasized that our views on which stu- 
dent preparation and test administration practi^ are ethical 
and unethical are not shared universally. Mehrens and 
Kaminski (1989) suggest that, to some extent, the acceptabil- 
ity of test preparation practices varies somewhat, depending 
on the purpose of testing. Our views, summarized in Table 
2, are based on the assumption that scores are and will con- 
tinue to be used to compare the educatiorud effectiveness of 
teachers, administrators, classes, schools, districts, states, 
and nations. 

E>espite the fact that some practices may be considered 
ethical and others unethical, it must be noted that even 
ethical practices are polluting if they are unevenly ad- 



Table2 

A Condnuum of Test Freparadon Activities 



Test preparation activity 


Degree of ethicality 


Training in testwiseness skills 


Ethical 


Checking answer sheets to make 
sure that each has been properly 
completed 


EthicaP 


Increasing student motivation to 
perform on the test through 
appeals to parents, students, and 
teachers 


Ethical 


Developing a curriculum based on 
the content of the test 


Unethical 


Preparing objectives based on items 
on the test and teaching 
accordingly 


Urtethical 


Presenting items similar to those on 
the test 


Unethical 


Using Scoring High or other score- 
boosting activities 


Unethical 


Dismissing low-achieving students on 
testing day to artificially boost test 
scores 


Highly unethical 


Presenting items verbatim from the 
test to be given 


Highly unethical 



^Ethtcai to the extent that the test publisher recommends it or to 
the extent that all schools, classes, and students being compared 
have the same service. 
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ministered within the unit of analysis being used for a par- 
ticular test score interpretation. If one school district uses ex- 
tensive test preparation programs for its students, is it valid 
to compare ^e test scores with those of a neighboring district 
where such test preparation does not take place? 

Evidence of polluting test preparation and administration 
practices has been accumulating. Mehrens and Kaminski 
(1989) recently reviewed research on the variey of test 
preparation practices listed in Table 2. They concluded that 
indeed many of these tactics to improve test scores work 
quite well, and they also recognized that many of these ac- 
tivities tend to spuriously inflate test scores. Surveys and in- 
terviews conducted in Arizona, a state which mandates an- 
nual standardized achievement testing of all students in 
Grades 2 through 11, reveal widespread use of questionable 
and blatantly unethical practices to boost test scores (Haas 
et al., 1989; Nolen et al., 1989; Smith, with others, 1990). 

Ethical Test Preparation Actmties 

Table 2 lists three classes of ethical activities to prepare 
students for stand^dized achievement tests. Training in 
testwiseness skills includes familiarizing students with the 
formats of answer sheets and test items and teaching general 
strategies for optimum performaiKe on multiple-choice tests. 
Samacki (1979) provides a useful review of the methods and 
the effectiveness of these methods on test performance. Most 
(although not all) of our respondents indicated that such 
training was routine in their school or district (Haas et al., 
1989; Nolen et al., 1989). Most respondents also reported 
various ethical strategies for increasing performance on these 
tests, such as demonstrating marking procedures (68.7%), 
sending notes home to parents about rest and nutrition 
(70.9%), encouraging attendance (92.6%), arni discussing the 
purpose and importance of the tests (75.0%). It was also 
reported that some school districts check answer sheets very 
carefully for proper completion, whereas others do not. 
Variations in handling answer sheets mean that some schools 
or districts are disadvantaged by sloppy marking. 

These surveys reveal that not all s^ool districts practice 
the same kinds of ethical test preparation or in equal 
amounts. Although such practices ^ould be eiKouraged and 
continued, as pointed out earlier in this article, the unequal 
application of these practices within a uiut of arudysis also 
pollutes test scores. 

Unethical Test Preparation and Administration Actioities 

Both sxuvey and interview responses indicated that certain 
unethical practices whidi inflate test scores without concur- 
rently raising students achievement level were common in 
their school or district (Haas et al., 1989; Nolen et al., 1989). 
The elementary level teachers repcirted that 41.2% of them 
used commer^ test preparation materiab which, in addi- 
tion to familiarizing students with test formats, teaches or 
reviews skills to be tested. Use of these materials may start 
several months prior to the test and may occur on a weekly 
or even daily basis. Mehrens and Kaminski (19W) have 
voiced the strongest objection to the ethics oi using these test 
preparation programs, and their review is a most compel- 
ling argument against the continuation of diis practice. 

In addition to commercial padcages, many educators 
reported that district curriculum and objectives had been 
"iJigned" to the particular achievement test given (Haladyna 
et al., 1989). More informal alignment also occurs: Nolen et 



al., (1989) reported 8.5% of secondary level teachers and 10% 
of elementary school teachers teach students items from the 
current year's test. 

We found evidence of nonstandard administration in both 
the survey and interview studies (Haas et al., 1989; Nolen 
et al., 1989). On the survey, 8% of the elementary schcx)l 
teachers reported deviations in reading test directions or in- 
creasing testing time, and more than a third stated that they 
weren't sure if they followed prescribed procedures exactly. 

When does nonstandard administration become cheating? 
Some interviewees discussed helping students select 
answers for some items during the test, and 14.2% of survey 
respondents offered rewards for test completion. Some in- 
terviewees reported that low-achieving students were ex- 
cused from tal^g the test by being disniissed or sent on field 
trips during test week. Such tachcs will increase or ensure 
a much higher performance than deserved. 

Undetected instances of cheating lead to false interpreta- 
tions of test performaiKe, one source of test score-pollution. 
Collective damage occurs because of cheating. Teaching the 
items from the test or changing students' responses on the 
answer sheet leads to gross misinterpretations of student 
achievement. Less obviously unethical (and so perhaps more 
insidious) achons such as aligning curricula to the content 
of a particular achievement test, or extensive practice with 
alternate forms or commercial preparation packages, have 
a wider effect and can lead to misintexpretadons of dif- 
ferences among schools and districts. By any reasonable 
standard, the extensiveness of score-polluting pracdces re- 
vealed in the reports reviewed here is staggering. As smciU 
as the percentages reported in various studies are, when 
muldplied by the millions of students taking these tests, the 
number of flawed results must be quite large. 

A Qinute for Test Score Pollution 

Many of the current uses of test scores have been character- 
ized as "high stakes" (Madaus, 1988) because tangible con- 
sequences depend upon test scores. When teachers' and 
principals' employment or salary advancement is linked to 
student perfoxmarKe on such tests, the stakes are Indeed 
high. 

As the number of uses for standardized achievement test 
scores has inoreased, so has the pressure to raise these scores. 
In a recent interview, George Madaus said: 

When the stakes are hi^ people are going to find wavs to 
have test scores go up. . . The school will look better, but 
the skill levds will not necessarily be going up. You mav 
have succeeded only in corrupting the inferepves vou 
wanted to make from the tests. (Brandt 1989. p 

This point has been made in a number of other contexts and 
times (e.g., Frederiksen, 1984). 

Educators themselves may vary considerabiv on what 
preparation and administration practices thev see as 
"cheating." Gonzales (1985, dted in Mehrens 4c Kaminski, 
1989) surveyed teachers in one schod district and found that 
11% did not consider teaching students by using a^tviaJ test 
items to be cheating. Thus some educators may be deluded 
into thmking foat any practioe that boosts scores is legmmate. 

Cheating occurs in situations where the consequences of 
success or failure are important and where confidence m a 
successful outcome is not complete. It is more likelv to oc- 
cur when indmduals are self-aware and comparing ihem- 
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selves to an external standard of performance (Malcolm & - 
1989 ) EarUer we established that the general public, the 
media, and poUtidans view standardized achievement test 
performance as an important indicator of academic eftec- 
^eness. It is clear that results have important consequences 
for individual teachers and administrators, as weU as for 
districts, and that results (for schooU or districts) are likely 



to be made public. . 

In high-stakes testing, many school personnel have an op- 
portunity to "optimize" their students' performance, 
without necessarily increasing achievement. Although most 
educators who engage in what we have caUed unethical 
preparation and administration practices probably do not 
corwider these activities as cheating, it is clear that the result 
is the same: polluted test scores. 



Until there is serious 
reform in the way schools 
prepare students to take 
standardized achievement 
tests, test results will con- 
tinue to misrepresent 
American public education 
and its accomplishments. 



The fact that our respondents caressed considerable 
doubt as to the validity of scores obtained under the preseru 
system (a finding similar to data reported by Smith ^ 
others, 1990) suggests that many are aware of the 
of test score poUution and the external conditions that lead 
to poUuting practices. As one anonymous teacher eloquently 
wrote (Haas et al., 1989): 



They (the tests] do not, in my opiniotv t^ect a studCTt s 

progress. I resent the amount of teaching time it takes from 
W year to prep for *e test I feel that in order for ^ sc^ 

to be competitive, I must use that t^, when it rauld be 
more effectively used for odier learning purposes man test 
perfornumce. (p. 96) 



The majority of our participants stated that 
pressure to increase student scores, but fewer than 20% felt 
the scores reflected a single year's learning. The extent o^ 

pressure to raise scores varies from school to schod and from 
district to district (a fact which in hse^ alinost guaranty bla- 
tant attempts to boost test scores). It is clear from their com- 
ments that many teachers and administrators feel they are 
"under the gun" (Haas et al., 1989). 

It is a shame that people fed the need to wahiatc one's 
achievement from marks on a bubble sheet. We as teachm 
are pressured to teach to the test. This is an absurd way to 
go a^t educating childien. I fed that if 1 am pr^ured any 

ILetodowdlontheTESr.thenlwilldoevery^lcari 

to make sure my kids do well. . even cheat. . . .Is the real 



world a bubble sheet in which we base our decisions and 
our moral values? (p. 128 ) 

The moral dilemma presented to teachers and ad- 
ministrators is more complex than it might seem. Smith, with 
others, (1990) wrote that teachers face the dissonance of 
matching their daily, intimate observation of each child with 
test results. This "interpretive context" is uiuque to the 
teacher of a classroom of children. Teachers report a consis- 
tent lack of belief in these test resxilts and in particular with 
how the results are used (Haas et al., 1989; Nolen et al., 1989; 
Smith with others, 1990). Although teachers and adminis- 
trators participate in etf orts to raise test scores, as one elemen- 
tary school principal commented, "we suffer a collective guilt 
in the process." 

Back to the Standards 

Messick (1975) stated that when one is considering the use 
of test scores for evaluation, not only the validity of the score 
interpretation but also the consequences of using the score 
in a particular way must be considered. The Standards for 
EdiuMtional and Ps^ologiad Testing makes clear that certain 
conditions must exist for public use of tests. The most vital 
of these conditions is that any test use must be supported 
by evidence attesting to the truthfulness of interpretations 
and the reasonableness of the use of these scores. The cur- 
rent overemphasis on standardized achievement test scores 
has created corwlitions under which they are badly polluted 
by test preparation activities, admiiustration practices, and 
other conditions. Further, scores are reported without the 
context which may be critical to their interpretation. As a 
result, the uses of test results listed in Table 1 become 
questionable. 

Until there is serious reform in the way schools prepare 
students to take standardized achievement tests, test results 
win continue to misrepresent American puHic education and 
its accomplishments. However, as long as test scores remain 
the sinde most important index of educational effectiveness, 
such reform is unlikely to take place. The educational 
research community must do its part to discourage poUuting 
practices, as weU as to educate the public about die problems 
inherent in oveireliance upon standardized achievement ted 
scores. More important, researchers, other educators, and 
policymakm must work together to develop means of 
evaluating educational effectiveness that accuratdy reprint 
a school or district's progress toward a broad range of im- 
portant educational gods. This work is beginning m a 

nurnber of states, iTKhiding Arizona, where our research was 



conducted. , . 

As new evalution tools are devdoped, however, their use 

must also be examined carefully lest the problems we have 

discussed here continue. New forms of sAooI achievement r j 
testing are stfll subject to the poUuting influence we have 
described. We must look not only at what these new m-^«^ 
struments teU us about student achievement but also at the 
ways in which students are prepared for and participate in tcaUcp 
the evaluation. As Messick, in his essay on mear^g and 
values in measurement and evaluation (1975), writes, 

■ L 1 j -1 - 

To judge the value of an outcorrre or end, one should under- c 

stand the nature of the processes or mearu that led to that 
end, as Dewey (1939) emphasized in his prinaple of the 
means-end continuum; it's not just that the mearw are ap- 
praised in terms of the ends they lead to, but ends 
praised in tcmns of the meairs tlwt produce them. (p. 963) 
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The use of educational testing in the United States has been criticized for its inequi^ 
table effects on different populations of students. Many assume that new forms of 
assessment will lead to more equitable outcomes, Linda Darling-Hammond argues in 
this article, however, that alternative assessment methods, such as performance-based 
assessment, are not inherently equitable, and that educators must pay careful atten- 
tion to the ways that the assessments are used. Some school reform strategies, for 
example, use assessment reform as a lever for external control of schools. These 
strategies, Darling-Hammond argues, are unlikely to be successful and the assess- 
ments are unlikely to be equitable because they stem from a distrust of teachers and 
fail to involve teachers in the reform processes, 

Darling-Hammond argues instead for policies that ensure top-down support for bottom- 
up reform, ** where assessment is used to give teachers practical information on student 
learning and to provide opportunities for school communities to engage in a recursive 
process of self-reflection, self-critique, self-correction, and self-renewal. Ultimately, then, 
the equitable use of performance assessments depends not only on the design of the assess- 
ments themselves, but also on how well the assessment practices are interwoven with the 
goals of authentic school reform and effective teaching. 

In recent years, the school reform movement has engendered widespread efforts 
to transform the ways in which students’ work and learning are assessed in 
schools. These alternatives are frequently called performance^based or “authen* 
tic” assessments because they engage students in “real-world” tasks rather than 
multiple-choice tests, and evaluate them according to criteria that are important 
for actual performance in a field of work (Wiggins, 1989). Such assessments 
include oral presentations, debates, or exhibitions, along with collections of 
students’ written products, videotapes of performances and other learning 
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occasions, constructions and models, and their solutions to problems, experi- 
ments, or results of scientific and other inquiries (Archbald &: Newman, 1988). 
They also include teacher observations and inventories of individual students* 
work and behavior, as well as of cooperative group work (National Association 
for the Education of Young Children [NAEYC],^ 1988), 

Much of the rationale for these initiatives is based on growing evidence that 
traditional norm-referenced, multiple-choice tests fail to measure complex cog- 
nitive and performance abilities. Furthermore, when used for decisionmaking, 
they encourage instruction that tends to emphasize decontextualized, rote-ori- 
ented tasks imposing low cognitive demands rather than meaningful learrring. 
Thus, efforts to raise standards of learning and performance must rest in part / 
on strategies to transform assessment practices. 

In addition, efforts to ensure that af/ students learn in meaningful ways re- 
sulting in high levels of performance require that teachers know as much about 
students and their learning as they do about subject matter. However, teachers’ \ 
understandings of students* strengths, needs, and approaches to learning are \ 

not well supported by external testing programs that send secret, secured tests / 

into the school and whisk them out again for machine scoring that produces ' 
numerical quotients many months later. Authentic assessment strategies can pro- 
vide teachers with much more useful classroom information as they engage 
teachers in evaluating how and what students know and can do in real-life per- 
formance situations. These kinds of assessment strategies create the possibility 
that teachers will not only develop curricula aimed at challenging performance 
skills, but that they will also be able to use the resulting rich information about 
student learning and performance to shape their teaching in ways that can prove 
more effective for individual students. 

Recently, interest in alternative forms of student assessment has expanded 
from ihe classroom-based efforts of individual teachers to district and statewide 
initiatives to overhaul entire testing programs so that they become more per- 
formance-based, Major national testing programs, such as the National Assess- 
ment of Educational Progress and the College Board’s Scholastic Assessment 
Tests (formerly the Scholastic Aptitude Tests), are also undergoing important 
changes. These programs are being redesigned so that they will increasingly 
engage students in performance t 2 isks requiring written and oral responses in 
lieu of multiple-choice questions focused on discrete facts or decontextualized 
bits of knowledge. 

However, proposals for assessment reform differ in several important ways: 

1) in the extent to which they aim to broaden the roles of educators, students, 
parents, and other community members in assessment; 2) in the extent to which 
they aim to make assessment part of the teaching and learning process, and use 
it to serve developmental and educational purposes rather than sorting and 
screening purposes; 3) in the extent to which they anticipate a problem-based 
interdisciplinary curriculum or a coverage-oriented curriculum that maintains 
traditional subject area compartments for learning; and 4) in the extent to which 
they see assessment reform as part of a broader national agenda to improve and 
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equalize educational opportunities in schools. Some see assessment reform as 
part of a broader agenda to strengthen the national educational infrastructure 
(the availability of high-quality teachers, curriculum, and resources) and to 
equalize access so that all students start from an equal platform for learning. 
Others, however, view performance-based assessment as a single sledgehammer 
foi change, without acknowledging other structural realities of schooling, such 
as vast inequalities in educational opportunities. 

These differences in approaches to assessment reform predict very different 
consequences for the educational system, and dramatically different conse- 
quences for those who have been traditionally underserved in U.S. schools 

students in poor communities, “minorities,” immigrants, and students with dis- 
tinctive learning needs. In this article, I argue in particular that changes in the 
forms of assessment are unlikely to enhance equity unless we change the ways in which 
assessments are used as well: from sorting mechanisms to diagnostic supports; from 
external monitors of performance to locally generated tools for inquiring deeply 
into teaching and learning; and from purveyors of sanctions for those already 

underserved to levers for equalizing resources and enhancing learning oppor- 
tunities. o rr 

The extent to which educational testing serves to enhance teaching and learn- 
ing and to support greater equality or to undermine educational opportunity 
depends on how a variety of issues are resolved. Among these are issues associ- 
aied with the nature of assessment tools themselves: 

- whether and how they avoid bias; 

how they resolve concerns about subjectivity versus objectivity in evaluating 
Student work; ° 

- how they influence curriculum and teaching. 

A second set of issues has to do with whether and how assessment results are 
used to determine student placements and promotions, to reinforce differential 
curriculum tracking, or to allocate rewards and sanctions to teachers, programs 
or schools. 

A final set of issues concerns the policies and practices that surround the 
assessment system and determine the educational opportunities available to stu- 
dents to support their learning. A fundamental question is whether assessment 
systems will support better teaching and transform schooling for traditionally 
underserved students or whether they will merely reify existing inequities. This 
depends on the extent to which they promote equity in the allocation of re- 
sources for providing education, supports for effective teaching practices, and 
supports for more widespread school restructuring. 
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Motivations for Assessment Reform 

The current movement to change U.S. traditions of student assessment in large- 
V scale and systemic ways has several motivations. One is based on the recognition 
that assessment, especially when it is used for decisionmaking purposes, exerts 
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powerful influences on curriculum and insuruction. It can “drive’* instruction in 
ways that mimic not only the content, but also the format and cognitive demands 
of tests (Darling-Hammond & Wise, 1985; Madaus, West, Hannon, Lomax, & 
Viator, 1992). If assessment exerts these influences, many argue, it should be 
carefully shaped to send signals that are consistent with the kinds of learning 
desired and the approaches to curriculum and instruction that will support such 
learning (Cohen & Spillane, 1992; O’Day & Smith, 1993). 

— A second and somewhat related motive for systemic approaches to assessment 
reform stems from the belief that if assessment can exert powerful influences 
on behavior, it can be used to change school organizational behavior as well as 
classroom work. The idea of using assessment as a lever for school change is not 
a new one: many accountability tools in the 1970s and 1980s tried to link policy 
decisions to test scores (Linn, 1987; Madaus, 1985; Wise, 1979). Unfortunately, 
these efforts frequently had unhappy results for teaching and learning generally, 
and for schools’ treatment of low-scoring students in particular. Research on 
these initiatives has found that test-based decisionmaking has driven instruction 
toward lower order cognitive skills. This shift has created incentives for pushing 
low scorers into special education, consigning them to educationally unproduc- 
tive remedial classes, holding them back in the grades, and encouraging them 
to drop out (Allington & McGill-Franzen, 1992; Darling-Hammond, 1991, 1993; 
Koreiz, 1988; Shepard & Smith, 1988; Smith, 1986). In addition, school incen- 
tives tied to test scores have undermined efforts to create and sustain more 
inclusive and integrated student populations, as schools are punished for accept- 
ing and keeping students with special needs and are rewarded for keeping such 
students out of their programs through selective admissions and transfer poli- 
cies. Those with clout and means “improve” education by manipulating the 
population of students they serve (Smith. 1986). Schools serving disadvantaged 
students find it increasingly hard to recruit and retain experienced and highly 
qualified staff when the threat of punishments for low scores hangs over them. 
Thus, such policies exacerbate rather than ameliorate the unequal distribution 
of educational opportunity. 

Nonetheless, a variety of proposals have recently been put forth that involve 
the use of mandated performance-based assessments as external levers for school 
change (Commission on Chapter 1, 1992; Hombeck, 1992; O’Day & Smith, 
1993). Even those who do not endorse such proposals share the view that assess- 
ment can promote change. Other proposals, raised from a different philosophi- 
cal vantage point and envisioning different uses of assessment, suggest the use 
of alternative classroom-embedded assessments as internal supports for school- 
based inquiry (Darling-Hammond & Ascher, 1990; Wolf & Baron, in press). 

A third reason for assessment reform addresses concerns about equity and 
access to educational opportunity. Over many decades, assessment results have 
frequently been used to define not only teaching, but also students’ opportuni- 
ties to learn. As a tool for tracking students into different courses, levels, and 
kinds of instructional programs, testing has been a primary means for limiting 
or expanding students’ life choices and their avenues for demonstrating com- 
petence. Increasingly, these uses of tests are recognized as having the unin- 



_A c 

J '! /' ■/ ^ 

u 

% • - 









8 




116 




Symposium: Equity in Educational Assessment 

LINDA DARLING-HAMMOND 




y /> •. . - . ' '} '• 

if (.r- ■■ 



...„ded consequence of limiting students’ access to further learning opportuni- 
ties (Darling-Hammond, 1991; Glaser, 1990; Oakes, 1985). 

Some current proposals for performance-based assessment view these new 
kinds of tests as serving the same screening and tracking purposes as more ira- 
tlitional tests (Commission on the Skills of the American Workforce, 1990; u- 
-,tc America 1991; National Center on Educauon and the Economy, 1989) . The 
, ‘resumption is that more “authentic” assessments will both motivate and sort 
Students more effectively. Others see a primary goal of assessment reform as 
transforming the purposes and uses of testing as well as its form and content. 
They argue for shifting from the use of assessment as a sorting device to its use 
■IS a tool for identifying student strengths and needs so that i^achers can a apt 
instruction more successfully (Darling-Hammond, Ancess, & Falk m press, 
riiser 1981, 1990; Kornhaber & Gardner, 1993). Given the knowledge now 
available for addressing diverse learning needs and the needs of today s society 
for a broadlv educated populace, the goals of educauon — and assessment 
are being transformed from deciding who will be permitted to become well-edu- 
cated to helping ensure that everyone will learn successfully. 

Clearly, the current press to reform assessment enuils many mouvations and 
manv possible consequences, depending on decisions that are made a out 
1) the nature of the “new” assessments; 2) the ways in which they are used; and 
3) the companion efforts (if any) that accompany them to actually improve edu- 

cation in the schools. 

In this article 1 outline the range of equity issues that arise with respect to 
testing generallv, and with respect to proposals for the development of new 
“authLtic” assessments specifically. 1 argue that the outcomes of the current 
wave of assessment reforms xvill depend in large measure on the extent to which 
assessment developers and users; 



- focus on both the quality and f^rimss of assessment su^tegies; 

- use assessments in ways that serve teaching ^djlearn^^ rather than sorting 
and selecting; 

- develop policies. that are congruent with (and respectful of) these assessment 
goals, as well as with assessment smategies and limiuuons; 

- embed assessment reform in broadej u:eforms to improve and equalize access 
to educational re sources and oppor tunities; 

- support the professional dwelopment of teachere along with the organiza- 
tional developiSim o f sc hools, so that assessment is embedded m teaching 
and learning, and is used to inform more skillful and adaptive teaching that 
enables more successful learning for all students. 



Uses and Consequences of Testing 



Historical Perspectives 

For over one hundred vears, standardized testing has been a tool used to exert 
control over the schooling process and to make decisions about educauonal 
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emiilements for students. Testing proved a convenient instrument of social con- 
trol for those superintendents in the late nineteenth century who sought to use 
tests as a means for creating the “one best system" of education (Tyack, 1974). 

t also proved enormously useful as a means of determining how to slot students 
lor more and less rigorous (and costly) curricula when public funding of edu- 

cauon and compulsory attendance vastly increased access to schools in the early 
twentieth century. ^ 

Given the massive increase in students, the limits of public budgets, and the 
relatively meager training of teachers, strategies were sought to codify curricu- 
lum and to group students for differential instruction. IQ tests were widely used 
as a measure of educational input (with intelligence viewed as the “raw material” 
lor schooling) to sort pupils so they could be efficiently educated according to 
their future roles in society (Cremin. 1961; Cubberly. 1919; Watson, in press) 

Frequently, they were used to exclude students from schooling opportunities 
altogether (Glaser. 1981). & kk 

Though many proponents argued that the use of these tests as a tool for 
tracking students would enhance social justice, the rationales for tracking — like 
those for using scores to set immigration quotas into the United States — were 
often frankly mouvated by racial and ethnic politics. Just as Goddard’s 1912 data 
proving that 83 percent of Jews. 80 percent of Hungarians. 79 percent of 
tahans. and 87 percent of Russians xvere “feebleminded” - were used to justify 
low immigrauon quota for those groups (Kamin. 1974). so did Terman’s test 
data prove that “[Indians. Mexicans, and Negroes] should be segregated in 
special classes. . . . They cannot master abstractions, but they can often be made 
efficient workers” (Terman. cited in Oakes. 1985. p. 36). Presumptions like these 
reinforced racial segregation and differential learning opportunities 

Terman found many inequalities in performance among groups’ on his 10 
test, which was adapted from Binet’s work in France. Most, but not all of them 
seemed to confirm what he. and presumably every “intelligent” person, already 
knexv: that ^nous groups were inherently unequal in their mental capacities 
However, when girls scored higher than boys on his 1916 version of the Stan- 
ford-Bmet. he revised the test to correct for this apparent flaw by selecting items 
to create parity among genders in the scores (Mercer. 1989). Other inequalities 
- beuveen urban and rural students, students of higher and lower socioeco- 
nomic status, native English speakers and immigrants. Whites antLBlacks - did 

not occasion such revisions, since their validity seemed patently obvious to the 
test-makers. 

The role of testing in reinforcing and extending social inequalities in educa- 
lonal opportunities has by now been extensively researched (Gould 1981- 
Kamin. 1974; Mercer. 1989; Oakes. 1985; Watson, in press) and tvidely acknow: 

1 , fallacies Gould describes: the fallacy of reification 

which allowed testers to develop and sell the abstract concept of intelligence as’ 
an innate, unitary, measurable commodity; and the fallacy of ranking which 
supported the development of strategies for quantifying intelligence in v4ys that 
would allow people to be arrayed in a single series against each other (Gould 
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1981). These two fallacies — recently debunked (though not yet dismantled) by 
understandings that intelligence has many dimensions (Gardner, 1983; Stern- 
berg, 1985) — were made more dangerous by the social uses of testing as a tool 
for allocating educational and employment benefits rather than as a means for 
informing teaching and developing talents. 



Negative Consequences of Standardized Testing 



Current standardized tests are widely criticized for placing test-takers in a pas- 
sive, reactive role (Wigdor & Garner, 1982). rather than one that engages their 
capacities to structure tasks, produce ideas, and solve problems. Based on out- 
moded views of learning, intelligence, and performance, they fail to measure 
students’ higher order cognitive abilities or to support their capacities to per- 
form real-world tasks (Resnick, 1987a; Sternberg, 1985). 

In a seminal paper on the past, present, and future of testing, Glaser (1990) 
makes an important distinction between testing and assessment. These two kinds 
of measurement have different purposes and different social and technical his- 
tories. olaser describes testiiig as aimed at selection and placement: it attempts 
to predict success at learning by “measur[ing] human ability prior to a course 
of instruction so that individuals can be appropriately placed, diagnosed, in- 
cluded or excluded (p. 2). Assessment, on the other hand, is aimed at gauging 
educational outcomes: it measures the results of a course of learning. What is 
important for testing is the instrument's predictive power rather than its content. 
What is important for assessment is the content validity of an approach — its 
ability to describe the nature of performance that results from learning. 

Recently, another validity construct has emerged: consequential validity^ which 
describes the extent to which an assessment tool and the ways in which it is used 
produce positive consequences both for the teaching and learning process and 
for students who may experience different educational opportunities as a result 
of test-based placements (Glaser, 1990; Shepard, 1993). This emerging validity 
standard places a much heavier burden on assessment developers and users to 
demonstrate that what they are doing works to the benefit of those who are 
assessed and to the society at large. The emergence of this standard has led many 
educators and researchers to question test-based program placements for stu- 
dents and to press for forms of assessment that can support more challenging 
and authentic forms of teaching and learning. Some test developers are just 
beginning to understand that the criteria against which their products are being 
evaluated are changing. 

For most of this centur\*, much of the energy of U.S. measurement experts 
has been invested in developing tests aimed at ranking students for sorting and 
selecting them into and out of particular placements. Standardized test devel- 
opers have devoted much less energy to worrying about the properties of these 
instruments as reflections of — or influences on — instruction (Wigdor & Gar- 
ner, 1982). As a consequence, the tests generally do not reflect the actual tasks 
educators and citizens expect students to be able to perform, nor do they stimu- 
late forms of instruction that are closely connected to development of perform- 
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ance abilities. Similarly, to date, though awareness levels are heightened, virtu- 
ally no attention has yet been paid to the consequences of test-based decisions 
in policy discussions about developing new assessment systems. 

These shortcomings of U.S. tests were less problematic when they were used 
as only one source of information among many other kinds of information about 
student learning, and when they were not directly tied to decisions about stu- 
dents and programs. However, as test scores have been used to make important 
educational decisions, their flaws have become more damaging. As schools have 
begun to teach to the tests,” the scores have become ever poorer assewments 
of students overall abilities, because class work oriented toward recognizing the 
answers to multiple-choice questions does not heighten students’ proficiency in 
aspects of the subjects that are not tested, such as analysis, complex problem- 
solving, and written and oral expression (Darling-Hammond & Wise, 1985- 
Haney & Madaus, 1986; Koretz, 1988). 




genuinely puzzled at requests to explain or defend their points of view.” The 
NAEP assessors explained that current methods of testing reading require short 
responses and lower level cognitive thinking, resulting in “an emphasis on shal- 
low and superficial opinions at the expense of reasoned and disciplined thought, 

. . [thus] it is not surprising that students fail to develop more comprehensive 
thinking and analytic skills” (NAEP, 1981, p. 5). / 

During the 1970s, when test-oriented accountability measures were instituted 
in U.S. Khools, there was a decline in public schools’ use of teaching methods 
appropriate to the teaching of higher order skills, such as research projects and 
laboratory work, student-centered discussions, and the writing of essays or 
themes (National Center for Education Statistics [NCES], 1982 p 83) Major ^ 
studies by Boyer (1983), Goodlad (1984), and Sizer (1985) documented the /I 
negative effects of standardized testing on teaching and learning in high schools 
while the disadvantage created for U.S. students by the rote learning stressed in 
U.S. standardized tests has been documented in international studies of achieve- 
ment (McKnight et al., 1987). 

The effects of basic skills test misuse have been most unfortunate for the 
students they were most intended to help. Many studies have found that students 
placed in the lowest tracks or in remedial programs — disproportionately low- 
income and minority students — are most apt to experience insuaiction geared 1 
only to multiple-choice tests, working at a low cognitive level on test-oriented 
tasks that are profoundly disconnected from the skills they need to learn. Rarely 
are they given the opportunity to talk about what they know, to read real books, 
to write, or to construct and solve problems in mathematics, science or other 
subjects (Cooper & Sherk, 1989; Davis. 1986; Oakes, 1985; Trimble & Sinclair, 
1986). In short, they have been denied the opportunity to develop the capacities 
they will need for the future, in large part because commonly used tests are so 
firmly pointed at educational goals of the p 2 isL 

Thus, the quality of education made available to many students has been 
undermined by the nature of the testing programs used to monitor and shape 
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their learning. If new performance-based assessments point at more challenging 
learning goals for all students, they may ameliorate some of this source of in- 
equality. However, this will be true only to the extent that teachers who serve 

these students are able to teach in the ways demanded by the assessments 

that is, in ways that support the development of higher order thinking and 
performance skills and in ways that diagnose and build upon individual learners’ 
strengths and needs. 

7 he Uses of Assessment Tools in Decisionmaking 

\s noted earlier, testing policies affect students’ opportunities to learn in other 
important ways. In addition to determining whether students graduate, tests are 
increasingly used to track students and to determine whether they can be pro- 
moted from one grade to the next Research suggests that both practices have 
had harmful consequences for individual students and for U.S. achievement 
generally. If performance-based assessments are used for the same purposes as 
traditional tests have been, the outcomes for underserved students are likely to 
be unchanged. 

Tracking In the United States, the process of tracking begins in elementary 
schools with the designation of instructional groups and programs based on test 
scores, and becomes highly formalized by junior high school. The result of this 
practice is that challenging curricula are rationed to a very small proportion of 
students. Consequently, few U.S. students ever encounter the kinds of curricula 
that most students in other countries typically experience (McKnight et al., 
1987). As Oakes (1986) notes, these assignments are predictable: 

One finding about placements is undisputed. . . . Disproportionate percentages of 
poor and minority youngsters (principally black and Hispanic) are placed in tracks 
for low-ability or non-college-bound students (NCES, 1985; Rosenbaum, 1980); 
further, minority' students are consistently underrepresented in programs for the 
gifted and talented. (College Board, 1985. p. 129) 

Students placed in lower tracks are exposed to a limited, rote-oriented cur- 
riculum and ultimately achieve less than students of similar aptitude who are 
placed in academic programs or untracked classes. Furthermore, these curricu- 
lar differences explain much of the disparity between the achievement of White 
and minority students and between those of higher and lower income levels (Lee 
&: Bryk, 1988; Oakes, 1985). In this way, the uses of tests have impeded rather 
than supported the pursuit of high and rigorous educational goals for all stu- 
dents. 

Grade Retention In addition, some U.S. states and local districts have enacted 
policies requiring that test scores be used as the sole criterion for decisions about 
student promotion from one grade to the next Since the student promotion 
policies were enacted, a substantial body of research has demonstrated that the 
effects of this kind of test-based decisionmaking are much more negative than 
positive. W^en students who were retained in grade are compared to students 
of equal achievement levels who were promoted, the retained students are con- 
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iistently behind on both achievement and social-emouonal measures (^Holmes 
& Matthews. 1984; Shephard & Smith. 1986). As Shephard and Sm.th put ,t. 
“Contrary to popular beliefs, repeating a grade does not help students gam 
ground academically and has a negative impact on social adjustment and self- 

esteem” (1986, p. 86). k t r Sn 

Furthermore, the practice of retaining students is a major contributor to in- 
creased dropout rates. Research suggests that being retained increases ‘^he o s ^ 
of dropping out by 40 to 50 percent. A second retenuon nearly doubles the nsk 
(Mann. 1987; see also Carnegie Council on Adolescent Development. 1989. s-^ 

sachusetts Advocacy Center. 1988; Wehlage. Rutter. Smith. Lesko. & Fernandez.^. 
1990) Thus, the policy of automatically retaining students based on their test- 
score performance has actually produced lower achievement for these students, 
lower self-esteem, and higher dropout rates for them and for the nation. 

Graduation Perhaps the ultimate test-related sancuon for students is denying a 
diploma based on a test score. The rationale for this pracuce is that students 
should show they have mastered the “minimum skills” needed for employment 
or future education in order to graduate. The assumption is that tests can a e- 
quately capture whatever those skills are. While this appears plausible in theory 
it is unlikely in reality, given the disjunction between muluple-choice tests of 
decontextualized bits of information and the demands of real jobs and a u t 
tasks (Bailey 1989; Carnevale. Gainer. & Meltzer. 1989; Resnick. 1987b). In fact. ^ 
research indicates that neither employability nor earnings are significantly af- 
fected by students’ scores on basic skills tests, while chances of employment and 
welfare dependency are tighUy linked to graduation from high school (Eckland. 
1980- Gordon & Sum. 1988; Jaeger. 1991). Thus, the use of tests as a sole deter- 
minant of graduation imposes heavy personal and societal costs, without obvious 

social benefits. 



Rewards and Sanctions Finally, a few states and districts have also tned to use 
student test scores to allocate rewards or sanctions to schools or teachers. Presi- 
dent Bush’s proposal for a National Test included a suggestion to allocate some 
federal funds based on schools’ scores on the “American Achievement Tests 
(U.S. Department of Education. 1991 ). An independent commission on Chapter 
1 has recently proposed, over the formal dissent of a number of its members, a 
rewards and sanctions system for Chapter I programs based on aggregate per- 
formance-based” test scores (Commission on Chapter 1. 1992). An analogous 
policy proposal has been enacted, though not yet implemented, for use with 
performance-based tests in the state of Kentucky. There, all schools that do not 
show specified percentage increases in student achievement scores each year will 
automatically suffer sanctions, which may include actions against staff. Those 
that meet the standards will be financially re%varded (Legislauve Research Com- 
mission. 1990. p. 21). . . 

Oblivious to the fact that schools’ scores on any measure are sensiuve to 
changes in the population of students taking the test, and that such changes can 
be induced by manipulating admission, dropouts, and pupil cl^sifications. the 
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policy will create and sustain a wide variety of perverse incentives, regardless of 
whether the tests are multiple-choice or performance-oriented. Because schools’ 
scores on any measure are sensitive to the population of students 
taking the test, the policy creates incentives for schools to keep out students 
whom they fear may lower their scores — children who are handicapped, limited 
English speaking, or from educationally disadvantaged environments. Schools 
where average test scores are used for making decisions about rewards and sanc- 
tions have found a number of tvays to manipulate their test-taking population 
in order to inflate artificially the school’s average test scores. These strategies j 
include labelling large numbers of low-scoring students for special education ' I 
placements so that their scores won’t "count" in school reports, retaining stu- 
dent in grade so that their relative standing will look better on “grade-equiva- 
lent” scores, excluding low-scoring students from admission to “open enroll- r.^>' 
ment" schools, and encouraging such students to leave schools or drop out 1 4,' 
(Allington & McGill-Franzen, 1992; Darling-Hammond, 1991, 1993; Koretz, ci-, 

1988; Shepard & Smith, 1988; Smith, 1986). 

Smith explains the widespread engineering of student populations that he | 
found in his study of a large urban school district that used performance stand- j 
ards as a basis for school level sanctions: 

Student selection pro\-ides the greatest leverage in the short-term accountability 
game. . . . The easiest way lo improve one s chances of winning is (1) to add some 
highly likely students and (2) to drop some unlikely students, while simply hanging 

on to those in the middle. School admissions is a central thread in the accountabiliiv 
fabric. (1986, pp. 30-31) ^ 

Needless to say, this kind of policy that rewards or punishes schools for ag- 
gregate test scores creates a distorted view of accountability, in which beating 
the numbers by playing shell games with student placements overwhelms efforts - 
to serve students’ educational needs well. Equally important, these policies fur- 
ther exacerbate existing incentives for talented staff to opt for school placements 
where studenu are easy to teach, and school stability is high. Capable staff are 
less likely to risk losing rewards or incurring sanctions by volunteering to teach 
where many students have special needs and performance standards will be more 
difficult to attain. This compromises even further the educational chances of 
disadvantaged students, who are already served by a disproportionate share of 
those teachers who are inexperienced, unprepared, and under qualified. 

Applying sanctions to schools with lower test score performance penalizes 
already disadvantaged students twice over: having given them inadequate schools 
to begin with, society will now punish them again for failing to perform as well 
as other studenu attending schools with greater resources and more capable 
teachers. This kind of reward system confuses the quality of education offered 
by schools with the needs of the students they enroll; it works against equity and 
integration, and against any possibilities for fair and open school choice, by 
discouraging good schools from opening their doors to educationally needy 
students. Such a reward structure places more emphasis on score manipulations 
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and student assignments or exclusions than on school improvement and the 
development of more effective teaching practices. 



Policies for Building an Equitable System 



Improving Teacher Capacity 



Because this nation has not invested heavily in teacher education and profes- 
sional development, the capacity for a more complex, student-centered ap- 
proach to teaching is not prevalent throughout the current teaching force. Fur- 
thermore, because teacher salaries and working conditions are inadequate to 
ensure a steady supply of qualified teachers in poor districts, low-income and 
minority students are routinely taught by the least experienced and least pre- 
pared teachers (Darling-Hammond, 1991; Oakes, 1990). Differences in achieve- 
ment between White and minority students can be substantially explained by 
unequal access to high-quality curriculum and instruction (Barr & Dreeben, 
1983; College Board, 1985; Darling-Hammond 8c Snyder, 1992a; Dreeben. 1987; 
Dreeben 8c Barr, 1987; Dreeben & Gamoran, 1986; Oakes, 1990). 

From a policy perspective, perhaps the single greatest source of educational 
inequity is this disparity in the availability and distribution of highly qualified 
teachers (Darling-Hammond. 1990). Providing equity in the distribution of 
teacher quality will be required before changes in assessment strategies result in 
more challenging and effective instruction for currently underserved students. 
This, in turn, requires changing policies and long-standing incentive structures 
in education so that shortages of well-prepared teachers are overcome, and 
schools sending poor and minority students are not disadvantaged by lower sala- 
ries and poorer working conditions in the bidding war for good teachers. Fun- 
damental changes in school funding are essential to this task. Since revenues in 
poor districts are often half as great as those in wealthy districts, state aid changes 
that equalize district resources are the first step toward ensuring access to quali- 
fied teachers (Darling-Hammond, in press). 

This crucial equity concern is finally gaining some attention in the rush to 
improve schools by testing. The recent report of the National Council on Edu- 
cation Standards and Testing (NCEST), while arguing for national performance 
standards for students, acknowledged the importance of “school delivery stand- 
ards” for educational improvements to occur. The Council’s Standards Task 
Force noted: 






/ 



If not accompanied by measures to ensure equal opportunity to learn, national 
content and performance standards could help widen the achievement gap between 
the advantaged and the disadvantaged in our society. If national content and per- 
formance standards and assessment are not accompanied by clear school delivery 
standards and polic)' measures designed to afford all students an equal opportunity 
to learn, the concerns about diminished equity could easily be realized. Standards 
and assessments must be accompanied by policies that provide access for all students 
to high quality resources, including appropriate instructional materials and well- 
prepared teachers. High content and performance standards can be used to chal- 
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Irni;e all siudenis with the same expectations, but high expectations will only result 
in common high performance if all schools provide high quality instruction de- 
signed to meet the expectations. (NCEST, 1992, pp. E12-E13) 




Delivery standards make clear that the governmental agencies that are impos- 
ing standards upon students are simultaneously accepting responsibility for en- 
Niiring that students will encounter the opfK>rtunities necessary for their success 
( Oarling-Hammond, 1993). Though this may seem a straightforward prereq- 
(lisiie for making judgments about students or schools, it marks an entirely dif- 
ferent approach to accountability in U.S. education than the one that has pre- 
dominated for most of the last two decades and is widespread today. Earlier 
approaches to outcomes-based accountability legislated minimum competency 
u Sts and sometimes punished schools or students with low scores without at- 
icnipting to correct the resource disparities that contributed to poor perform- 
ance in the first place. 

( . ^ Ensuring that all students have adequate opportunities to learn requires en- 
^ hancing the capacity of all teachers — their knowledge of students and subjects, 
and their ability to use that knowledge — by professionalizing teaching. This 
means that teacher education policies must ensure that all teachers have a 
stronger understanding of how children learn and develop, how assessment can 
be used to evaluate what they know and how they learn, how a variety of cur- 
ricular and instructional strategies can address their needs, and how changes in 
school and classroom organization can support their growth and achievement. 

Such teacher capacities are also important for supporting the promise of 
authentic assessment to enable richer, more instructional ly useful forms of evalu- 
ation that are also fair and informative. A major reason for the advent of exter- 
nally controlled highly standardized testing systems has been the belief that 
teachers could not be trusted to make sound decisions about what students know 
and are able to do. The presumed “objectivity” of current tests derives both from 
the lack of reliance upon individual teacher judgment in scoring and from the 
fact that test-takers are anonymous to test-scorers (hence, extraneous views about 
the student do not bias scoring). 

Of course, many forms of bias remain, as the choice of items, responses 
deemed appropriate, and content deemed important are the product of cultur- 
ally and contextually determined judgments, as well as the privileging of certain 
ways of knowing and modes of performance over others (Garcia Sc Pearson, in \ 
press; Gardner, 1983; Sternberg, 1985; Wigdor & Gamer, 1982). And these forms ^ 
of bias are equally likely to plague performance-based assessments, as the selec- 
tion of tasks will rest on cultural and other referents, such as experiences, terms, 
and exposures to types of music, art, literature, and social experiences that are 
differentially accessible to test-takers of different backgrounds. 

If assessment is to be used to open up as many opportunities as possible to as 
many students as possible, it must address a wide range of talents, a variety of 
life experiences, and multiple ways of knowing. Diverse and wide-ranging tasks 
that use many different performance modes and that involve students in choos- 
ing ways to demonstrate their competence become important for this goal (Gor- 
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don. no date; Komhaber & Gardner. 1993). Substantial teacher and student 
involvement in and control over assessment strategies and uses are critical if 
assessment is to support the most challenging education possible for every stu- 
dent. taking full account of his or her special talents and ways of knowing. As 
Gordon puts it: 

The task is to find assessment probes which measure the same criterion from 
contexts and perspectives which reflect the life space and values of the learner. . . . 
Thus options and choices become a critical feature in any assessment system created 
to be responsive to equity, just as processual description and diagnosis become 
central purposes, (no date, pp. 8-9) 

The objective of maintaining high standards with less standardization will 
dem 2 tnd teachers who are able to evaluate and eliminate sources of unfair bias 
in their development and scoring of instructionally embedded assessments, and 
who can balance subjectivity and objectivity, using their subjective knowledge of 
students appropriately in selecting tasks and assessment options while adhering 
to common, collective standards of evaluation. These same abilities will be cru- 
cial for other assessment developers. In many respects, even greater sensitivity 
to the sources of bias that can pervade assessment will be needed with forms 
that frequently eliminate the anonymity of test-takers, drawing more heavily on 
interpersonal interaction in tasks and on observations on the part of teachers. 



^^Top'Down Support for Bottom-Up Reform ” 

The need for greater teacher knowledge and sensitivity in developing and using 
authentic assessments in schools will cause some to argue that they should not 
be attempted; that externally developed and scored “objective” tests are safer for 
making decisions because local judgement is avoided. However, the argument 
for authentic assessments rests as much on a changed conception of the tises of 
assessment as on the form in which assessment occurs. Rather than being used 
largely to determine how students rank against one another on a single, limited 
dimension of performance so as to determine curriculum or school placements 
of \^rious kinds, many reformers hope that assessment can be used to inform and 
improve teaching and learning. 

In this view, assessment should be integrally connected to the teaching and 
learning process so that students’ strengths and needs are identified, built upon, 
and addressed. School-wide assessments should continually inform teachers’ col- 
lective review of their practice so that improvements in curriculum, instruction, 
and school organization are ongoing. Thus, students should actually learn more 
as a result of assessment, rather than being more precisely classified, and schools 
should be able to inquire into and improve their practices more intelligendy, 
rather than being more rigidly ranked. Assessment should increase the overall 
amount of learning and good practice across all schools, rather than merely 
measuring how much of a nonexpanding pool of knowledge is claimed by dif- 
ferent students and schools. 

If authentic assessment is to realize its potential as a tool for school change, 
however, policies must enable assessments to be used as a vehicle for student. 
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BIBLIOGRAPHY 

Selected Abstracts from the ERIC 
Educational Resources Database 




ERIC Clearinshouse on 
Reading, English, and Communication 

Indiana University 
Bloomington, Indiana 






You may also wish to perform your own ERIC database search, to retrieve the most current 

L J? M Education library; it may also be 

variable to you online through your university computing system. 



In the following bibliography, we have selected some recent relevant articles that you may 
wish to read for your further knowledge, or to use in a Distance Education 

Apphcation/Research Project. ERIC abstracts are easy to read, once you are used to the 
system, which is detailed below. 



Sample ERIC Abstract 

ore t at this abstract has an EJ accession number, which means that tha work abstracted is a journal article. 



Eric Accession Number — 
ideniilicaiion number sequeniially 
assigned lo articles as they are 
F>focessed. 



Article Title 

Aulhor^s) 

Reprint Availability 
Descriptive Note 



Major and Minor Descriptors 

subject terms found in the 
Thesaurvs of EfilC Descriptors that 
characterize substantive content. 
Only the major terms (preceded by 
an asterisk) are printed in the 
Subject index of Current index to 
Joumats in Education (CUE). 



AnrK)tation 



Annotator's Initials 



EC6062S7 

Family-Centered Techniques: Integrating 
Enablement into the IFSP Process. Andrews. 
Mary A.; Andrews, James R. Journal of Childhood 
Comm li/i /car; 0/1 Disorders. vI5 nl p41— d6 1993 
(Reprint: UMI) 

Note: Theme Issue: Service Delivery to Infants and 
Toddlers. Current Perspectives. 

ISSN; 0735-3170 

Descriptors. Child Rearing; “Communication 
Disorders: ‘Early Intervention; 



' Clearinghouse Accession 
Number 



Journal Title 



Volume No., Issue No.. Pages 
Publicaton Date 



ISSN 

(iniernaiional Standard Senai 
Number) 



•Family Involvement; Individual Development; 

ObjcCTives; Parenting Skills; Skill Development; 

•Teamwork; Young Children 

IdentiFiers; •Enabler Model; Family Needs: 

Individualized Family Service Plans 

This anicle describes techniques, used in a family- 
centered cariy intervention project, that both assist in 
accomplishing the goals of the Individualized 
Family Service Plan process and create opportunities 
for families to display (heir present competencies 
and acquire new ones to meet the needs of (heir 
children wiih communication disorders. 
(Author/JDD) 



Major and Minor Identifiers — 
terms found in the Identifier 
Authority List that characterize 
proper names of concepts not yet 
represented by descriptors. Only the 
major terms (preceded by an 
asterisk) are printed in the Subject 
Index of Current UxSex to Journals in 
Education. 



fjQIfl; The fomtat of an ERIC Journal 
Artido resume will vary according to 
the source from which the database 
Is accessed. The above format is 
from the printed Irxfex. Carront index 
to Journals in Education, 



GETTING COPIES OF THE ITEMS DESCRIBED IN THE ERIC DATABASE; 



The l^ens described in bhe ERIC database have either an "ED” or an 
"EJ" number in the first field. About 98% of the ED items can be 
found in the ERIC Microfiche Collection. The ERIC Document 
Reproduction Service (EDRS) in Alexandria, Virginia can produce either 
microfiche or paper copies of these docximents for you. Check the 
accompanying list of ERIC Price Codes for their current prices. 

Alternatively, you may prefer to consult the ERIC Microfiche 
Collection yourself before choosing documents to copy. Over 600 
libraries in the United States subscribe to this collection. To find 
out which libraries near you have it, you are welcome to call the ERIC 
Clearinghouse on Reading -and Communication Skills at (812) 855-5847. 
Most such libraries have equipment on site for inexpensive production 
of paper copies from the fiche. 



For those few ED-numbered items not found in the Microfiche 
Collection, check the availability (AV) field of the citation to get 
information about the author, publisher, or other distributor. 

Items with an EJ number in the first field of the citation are journal 

copyright restrictions, ERIC cannot provide copies 
of these articles. Most large college or university libraries 
subscribe to the journals in which these articles were published, and 
the general public can read or copy the articles from their 
collections. Should you want copies of articles which appeared in 
journals not owned by your nearest university library, arrangements 
usually can be made via interlibrary loan; there frequently is a 
nominal charge for this, which is set by the lending libra^. if you 
a faculty member, staff member, or student at the university, just 
ask at your library's reference desk. 

categories of users, most universities cannot provide 

interlibrary services. However, public libraries which are there to 

residents typically are hooked into statewide lending 

networks designed to ensure that all state residents have access to 
materials of interest. Ask your local public librarian about 
interlibrary loan policies^ charges, etc. 

I™® professional reprint services which have obtained 

peraission from some journals to sell article copies. These are 
University Microfilms International (Article Clearinghouse, 300 North 

732-0616), and the 

Information (Original Article Tear Sheet 

Philadelphia, Pennsylvania 19104 (800) 

publication, UMI charged $10.75 per 
article regardless of length, and ISI charged $9.50 for the first ten 

HowoS' $2.00 for each additional ten pages or fraction thereof, 

ever, please check with them for current prices before ordering. 




AN: EJ430424 

AU: Watklnson,-Anne 

Tl: Primarily Assessing. 

PY: 1991 

JN: Education-in-Sclence; n141 p8-9 Jan 1991 
AV: UMI 

DER: Curriculum-Evaluation; Elementary-Education; 
Foreign-Countries; Science-Education; Standardized-Tests; 
Standards- 

DEM: ‘Academic-Achievement; 
‘Elementary-School-Science; ‘Evaluation-Methods; 
‘Observation-; ‘Student-Evaluation 
AB: The Idea of assessment driving the curriculum and the 
Importance of assessing students constantly instead of 
using only standardized tests are discussed. Informal 
observations used by teachers as a way of assessing 
student progress is emphasized. The United Kingdom’s 
National Curriculum assessment program are described. 
(KR) 

AN: EJ430344 

AU: Perrone,-Vito 

TI: On Standardized Testing. 

PY: 1991 

JN: Childhood-Education; v67 n3 p131-42 Spr 1991 
AV: UMI 

DER: Achievement-Tests; Elementary-Education; 
Performance-Tests; Position- Papers; Student-Development; 
Teacher-Made-Tests; Teacher-Student-Relationship; 
Teaching-Methods; Writing-Evaluation 
DEM: ‘Educational-Testing; ‘Elementary-School-Students; 
‘Standardized-Tests; ‘Student-Evaluation; ‘Test-Use 
AB: Reviews the history and uses of standardized testing, 
and suggests alternative methods of assessment. States the 
belief that testing exerts undue pressure on children, 
provides no useful Information on individual children, and 
limits educational possibilities. Discusses the Association for 
Childhood Education International position paper which 
states that all testing of young children in preschool and 
grades K-2 should cease. (BC) 

AN: EJ408022 
AU: Norrls,-Stephen-P. 

Tl: Can We Test Validly for Critical Thinking? 

PY: 1989 

JN: Educational-Researcher; v18 n9 p21-26 Dec 1989 
AV: UMI 

NT: Theme issue with title, "Educational Assessment." 

DER: Elementary-Secondary-Education; Evaluation-; 
Problem-Solving 

DEM: ‘Credibility-; ‘Critical-Thinking; 

‘Multiple-Choice-Tests; ‘Test-Validity; ‘Verbal-Tests 
AB: Discusses the generalizabillty of critical thinking and the 
disposition to think critically. Argues that verbal reports of 
examinees’ thinking on multiple-choice tests can explain the 
reasoning behind their answers and, thus, can be used to 
assess the inability to make credibility judgments. (FMW) 

AN: EJ519194 

AU: Shepard,-Lorrie-A.; Bliem,-Carribeth-L. 

Tl: Parents’ Thinking about Standardized Tests and 
Performance Assessments. 

PY; 1995 

JN: Educational-Researcher; v24 n8 p25-32 Nov 1995 
AV: UMI 



DER: Elementary-Secondary-Education; Parent-Attitudes; 
Public-Opinion; Test-Construction; Test-Results 
DEM: ‘Achievement-Tests; ‘Educational-Assessment; 
‘Parents-; ‘Standardized-Tests; ‘Test-Use 
AB: Parental attitudes about standardized tests and 
performance assessments were studied through Interviews 
with 60 parents or parent dyads In schools Involved in a 
project to develop new assessments and through 
questionnaires completed by parents from control schools. 
Parent approval of standardized tests did not imply 
disapproval of performance assessment. (SLD) 

AN: EJ517092 

TI: Ready-Reference Folder. 

PY: 1995 

JN: Learning; v24 n1 p39-42 Aug 1995 
AV: UMI 

DER: Elementary-Secondary-Education; 

Portfolios-Background-Materials 

DEM: ‘Evaluation-Methods; ‘Portfolio-Assessment; 

‘Standardized-Tests; ‘Student-Evaluation 

AB: A tear-out folder provides information on alternative 

student assessment versus standardized testing, describing 

what alternative assessments do and do not consist of and 

involve. Information on student assessment statewide and 

nationwide is presented along with a discussion of the costs 

of alternative assessment. All Information is presented In 

color-coded charts. (SM) 

AN: ED389734 

AU: Shepard,-Lorrie-A.; B!eim,-Caribeth-L. 

Tl: An Analysis of Parent Opinions and Changes In 
Opinions Regarding Standardized Tests, Teacher's 
Information, and Performance Assessments. 

CS: National Center for Research on Evaluation, Standards, 
and Student Testing, Los Angeles, CA. 

PY: 1995 

NT: 63 p.; Portions of the report presented at the Annual 
Meeting of the American Educational Research Association 
(1993). 

PR: EDRS Price - MF01/PC03 Plus Postage. 

DER: Beliefs-; EIementary-Secx)ndary-Educatlon; 
Parent-Teacher-Conferences; Parent-T eacher-Cooperation; 
Questionnaires-; Report-Cards 
DEM: ‘Educational-Assessment; ‘Parent-Attitudes; 
‘Parents-; ‘Standardized-Tests; ‘Test-Use 
AB: Parent opinions about standardized tests and 
performance assessments were examined systematically. 
Mutually exclusive but randomly equivalent stratified 
samples from schools participating in a study of 
performance assessment and control schools were used to 
measure change in parent opinion over time. Approximately 
one-third of parents (n=105) completed questionnaires at the 
beginning of the school year, one-third completed them at 
the end of the year (similar sample), and the remaining third 
supplied interview samples (n=33 and n=27, respectively). 
Results demonstrated that parents' favorable ratings of 
standardized national tests did not imply a preference for 
this type of educational assessment over other types of 
assessment for measuring student or school progress. 
Parents considered report cards, hearing from the teacher, 
and seeing graded samples of student work as more 
informative than standardized tests, and they wanted 
comparative information to measure their own child’s 
progress. When parents had a chance to look at 
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performance assessments through the year, they endorsed 
their use for district purposes and preferred them for 
classroom use. Survey data like the Gallup Poll showing 
widespread approval of standardized tests should not be 
taken to mean that parents are opposed to other forms of 
assessment. Appendixes contain the parent questionnaire 
and the interview protocol. (Contains 3 figures, 17 tables, 
and 9 references.) (SLD) 

AN: ED388484 

AU: Estrin,-Elise-Trumbull; Nelson-Barber,-Sharon 
TI: Issues In Cross-Cultural Assessment; American 
Indian and Alaska Native Students. Knowledge Brief, 
Number Twelve. 

CS: Far West Lab. for Educational Research and 
Development, San Francisco, Calif.; Native Education 
Initiative of the Regional Educational Labs. 

PY: 1995 
NT: 9 p. 

PR: EDRS Price - MF01/PC01 Plus Postage. 

DER: American-Indians; Classroom*Communication; 
CulturaFContext; Culture^Conflict; Educational-Environment; 
Educational-Strategies; Elementary-Secondary-Education; 
Epistemology-; Evaluation-Problems 
DEM: *Alaska-Natives; •American-Indian-Education; 
•Cultural-Differences; •Culturally-Relevant-Education; 
•Instructional-Effectiveness; *Student-Evaluation 
AB: This brief focuses on assessment issues for American 
Indian and Alaska Native (collectively referred to as 
"Native") students, as well as other pedagogical issues 
related to improved teaching and educational outcomes. 
Although traditional Native educational strategies emphasize 
cooperation, experiential learning, and reflection. Native 
students continue to be at a disadvantage in the classroom. 
The reasons lie in several intersecting realities: troubled 
historical relations between tribes and the federal 
government affecting Native schooling, ongoing educational 
practices that ignore or devalue cultural ways of knowing, 
and the dearth of American Indian and Alaska Native 
teachers. Understanding the school performance of Native 
students requires a sociocultural perspective that takes into 
account differences between community and school in 
social and cultural context, the unconscious nature of these 
contexts, effects on student learning and organization of 
knowledge, and implications for effective instructional styles 
and student evaluation. Despite supportive federal 
legislation, a repertoire of culturally specific instruments to 
assess Native student performance does not exist. 
Standardized norm-referenced tests present such difficulties 
as inappropriate content, time pressures, reliance on verbal 
information, basic premises of multiple-choice testing, and 
alien nature of formal on-demand testing. Indeed, 
achievement tests can be seen as merely indices of the 
student's acculturation to Western cultur^ knowledge and 
conventions for displaying knowledge. More culturally 
responsive assessment incorporates content reflecting local 
contexts and experiences, uses procedures that reflect local 
ways of thinking and learning, and provides students with 
options. Other concerns related to the question of whose 
standards are appropriate, proper interpretation and use of 
test data, and the value of alternative assessments. (SV) 

AN: ED386707 

AU: Holland,-Kathleen, Ed.; And-Others 

TI: Alternative Perspectives In Assessing Children’s 

Language and Literacy. 

PY: 1994 
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AV: Ablex Publishing Corporation, 355 Chestnut Street, 
Norwood, NJ 07648 (cloth: ISBN-0-89391 -864-4, $49.50; 
paperback: ISBN-0-89391 -914-4, $24.50). 

NT: 235 p. 

PR: Document Not Available from EDRS. 

DER: Elementary-Secondary-Education; Ethnography-; 
Evaluation-Methods; Language-Arts; Language-Skills; ' 
Sociolinguistics- 

DEM: "Child-Language; "Literacy-; "Reader-Response; 
"Student- Evaluation; "Writing-Evaluation 
AB: Suggesting many ways (not just one way) to understand 
what children are doing with language and literacy, this book 
presents essays that address the need for alternative 
perspectives as well as anthropological, 
sodo-psycholinguistic, and reader response perspectives in 
assessing children's language and literacy. After 
"Introducation: What Is an Alternative?" by editors, David 
Bloome, Kathleen Holland, and Judith Solsken, part 1, which 
discusses the need for alternative perspectives, includes 
essays: (1) "Language, Culture and the Implications of 
Assessment" (Karla Holloway); and (2) Towards an 
Alternative View of Writing Assessment" (Loren S. Barritt). 
Essays in the second part, which discusses anthropological 
perspectives, are: (3) "The Language of Testing: An 
Ethnographic-Sociolinguistic Perspective in Standardized 
Tests" (Catherine Emihovich); (4) "You Can1 Get There from 
Here" (David Bloome); and (5) "Assessing Students as 
Members of a Literate Community" (Beth Gildin Watrous 
and Jerri Willett). A discussion follows: "Assessment In My 
World" (Maryann Jennings). Essays in the third part, which 
discusses socio-linguistic perspectives, are: (6) "Alternative 
Language Assessment: Communicating Naturally with 
Students in Assessment Contexts" (Helen B. Slaughter); (7) 
Assessing the Written Language Abilities of Beginning 
Writers" (Jo-Anne R. Wilson Keenan); and (8) "Looking at 
Their Own Words: Students’ Assessment of Their Own 
Writing" (Susan Benedict). A discussion follows: "Making 
Assessment a Process" (Nina Tapper and Rocio Costa). 
Essays in the fourth part on reader response perspectives 
are: (9) "Children's Response to Literature: Isn’t It about 
Time We Said Good-Bye to Book Reports and Literal Oral 
Book Discussions?" (Kathleen Holland and Susan Lehr); 

(10) "Assessing Literary Understanding through Oral 
Language" (Joanne M. Golden); and (11) "Children’s Group 
Discussions of Literature: Fertile Ground for Informal Oral 
Language Assessment" (Lenore Carlisle). A discussion 
follows: "Unleashing the Potential of Children’s Responses 
to Literature" (Leslie Shaw, Deborah G. Jacque, and Cheryl 
L. Taylor). (RS) 

AN: EJ505556 

AU: Schommer,-Marlene 

TI: Voices In Education on Authentic Assessment. 

PY: 1995 

JN: Mid-Westem-Educational-Researchen v8 n2 p13-14 Spr 
1995 

DER: Costs-; Educational-Practices; Educational-Testing; 
Elementary-Secondary-Education; 

National-Competency-T ests 

DEM: "Educational-Change; "Evaluation-Methods; 

"Program-Implementation; "Student-Evaluation; 

"T eacher- Attitudes 

AB: Six prominent educators discuss barriers to achieving 
quality authentic assessment and the feasibility of 
implementing authentic assessment on a national basis. 
Points out the importance of using multiple assessments, 
the lack of a definition of authentic assessment, problems in 
developing quality assessments, and the immense cost of 
implementing authentic assessment nationally. (LP) 
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AN: ED384610 

AU: Galloway,-Dan; Schwartz, -Wendell 

Tl: Designing More Effective Grouping Practices at the 

High School Level. 

PY: 19Q4 

NT: 30 p.; Paper presented at the Annual Meeting of the 
Association for Supervision and Curriculum Development 
(49th, Chicago, IL, March 19-22, 1994). 

PR: EDRS Price - MF01/PC02 Plus Postage. 

DER: At-Risk-Persons; Cooperative-Learning; 
Educational-Diagnosis; Educational-Improvement; 
Heterogeneous-Grouping; High-Schools; 
Homogeneous-Grouping; School-Restructuring; 
Self-Concept; Student-Motivation; 

Teacher- Expectations-of-Students; Tutorial- Prog rams 
DEM: ‘Educational-Change; 
‘Grouping-Instructional-Purposes; ‘Labeling-of-Persons; 
‘Student- Evaluation; ‘Student- Placement 
AB: Efforts at one high school to reconsider Its practices of 
ability grouping and explore alternative assessment and 
grouping practices are described. Assessment of the 
schools* practices found that students in lower ability groups 
had a less stimulating curriculum, fewer positive role 
models, lower motivation, lower expectations for 
themselves, and worked with teachers who also held lower 
expectations for them. When mobility did take place 
between ability levels, It was more often downward than 
upward. The use of national standardized placement tests 
was replaced by teacher-designed, criterion-referenced 
assessment tools, resulting in significantly different balances 
of placements. A pilot program was launched to replace a 
remedial composition course with participation in regular 
level classes supplemented by ongoing lunch hour tutoring 
in composition, resulting in improved grades for participants. 
The success of this program led the school to eliminate 
lower ability levels in other content areas, and to modify 
curriculum in remaining lower-level courses. A variety of 
modifications were implemented to support heterogenous 
grouping, Including expanded use of cooperative learning 
and classroom workshops. Staff development is seen as 
essential to the future of these modifications. (PB) 

AN: ED382670 
AU: Burstein, -Leigh 

Tl: Performance-Based Assessment for Accountability 
Purposes: Taking the Plunge and Assessing the 
Consequences. 

CS: National Center for Research on Evaluation, Standards, 
and Student Testing, Los Angeles, CA. 

PY: 1994 

NT: 15 p.; A version of a paper presented at the Annual 
Meeting of the American Educational Research Association 
(Chicago, IL, April 1991). 

PR: EDRS Price - MF01/PC01 Plus Postage. 

DER: Costs-; Educational-Change; Educational-Planning; 
Psychometrics-; Test-Construction; Test-Items 
DEM: ‘Accountability-; ‘Educational-Assessment; 
‘Multiple-Choice-Tests; ‘StandardIzed-Tests; 
‘Testing-Problems; ‘Test-Use 
AB: Issues In alternative assessment for accountability 
purposes are discussed. Most new forms of performance 
assessment are linked in the literature, but all alternative 
forms of assessment do not have the same attributes in 
terms of technical and feasibility criteria. Tradeoffs in the 
validity of inferences that can be drawn from alternative 
assessments and typical multiple-choice assessments must 
be assessed in light of the costs of performance 
assessment. There are problems to solve before 
implementing performance assessment including a host of 



technical and practical problems in terms of its use in 
large-scale assessment. Curriculum forces and the 
educational policy community are committed to shifting from 
assessment solely through multiple-choice to assessments 
that represent more important knowledge and skills closer to 
the attributes of interest. The question is not whether we will 
have performance assessment, but rather when. The way to 
move to performance assessment for accountability 
purposes is to tackle change in stages, planning for 
implementation and including planning for reporting results. 
Performance assessment Is going to be a part of the 
response to the new American pluralism and to the 
increasingly global society. (Contains 12 references.) 

AN: ED380786 

AU: Stallman,-Anne-C.; And-Others 

Tl: Alternative Approaches to Vocabulary Assessment 

Technical Report No. 607. 

CS: Center for the Study of Reading, Urbana, IL. 

PY: 1995 
NT: 20 p. 

PR: EDRS Price - MF01/PC01 Plus Postage. 

DER: Academic-Achievement; Concurrent-Validity; 
Intermediate-Grades; Reading-Research; Reliability-; 
Standardized -Tests; Test- Validity 
DEM: ‘Vocabulary-Development 

AB: Interviews with children about their knowledge of a set 
of words was used to examine the concurrent validity of 
three paper-and-pencil measures of knowledge of these 
words-a standardized vocabulary test and two 
experimenter-designed tests. One experimenter-designed 
test, the Levels test, had three multiple-choice items per 
word that targeted three different levels of word knowledge. 
The other was a forced-choice contexts test with five items 
per word, each requiring a decision about whether the word 
was used appropriately in the context. Subjects were 50 
students from two heterogeneously grouped fifth-grade 
classrooms in a midwestem school district. All three 
paper-and-pencil measures showed acceptable levels of 
reliability. When subjects were used as the unit of analysis, 
the interview was more highly correlated with the 
standardized test and the Levels test than with the Contexts 
test. When the word was used as the unit of analysis, the 
interview correlated more highly with the Contexts and the 
Levels test than with the standardized test. These results 
are interpreted as indicating that standardized measures are 
more effective at discriminating among students upon the 
basis of their overall ability, but less accurate as measures 
of how much the students know about particular words. The 
Contexts test has the advantages of the highest reliability of 
the three measures, as well as the greatest instructional 
validity. (Contains 25 references and 7 tables of data.) 
(Author/RS) 

AN: ED380508 
AU: Davis, -Wesley 

Tl: Alternative Assessment: Facts and Opinions. 

CS: Florida Educational Research Council, Inc., Sanibel. 

PY: 1994 
JN: 

Florida-Educational-Research -Counci I- Research-Bulletin; 
v25 n4 pi -32 Sum 1994 

AV: Florida Educational Research Council, Inc., P.O. Box 
506, Sanibel, FL 33957 ($4; annual subscription, $15; 10% 
discount on 5 or more copies). 

NT: 34 p. 

PR: EDRS Price - MF01/PC02 Plus Postage. 

DER: Educational-Change; Educational-Improvement; 
Elementary-Secondary-Education; Literature-Reviews; 
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Norm-Referenced-Tests; Standardized'Tests; Teacher-Role; 
Test-Use 

DEM: •Cost-Effectiveness; •Educational-Assessment; 
•Evaluation-Methods; •Opinions-; •Student-Evaluation; 

•T est-Construction 

AB: An attempt is made to separate facts from opinions 
based on review of a representative sample of contemporary 
writings on alternative assessment. A summary listing of 15 
statements perceived to be factual is offered, followed by 
opinions of the author. These items cover: (1) the historical 
background and origins of alternative assessments; (2) their 
current intent, focus, and emphasis; (3) their technical 
problems and limitations; (4) the potential impact for change 
these procedures may have on instruction and 
student-teacher relationships; (5) other possible 
consequences of changes; (6) the expanded role of 
teachers in implementation; (7) the most significant 
contribution alternative assessment might make for 
students; and (8) projected cost factors. It is suggested that 
changing the instructional process for the better may well be 
the major contribution of alternative assessment. 

Large-scale norm-referenced standardized tests are here to 
stay, and cost factors may mean that alternative 
assessments are most useful in the individual classroom. 
One table summarizes facts about alternative assessment. 
(Contains 44 references.) (SLD) 

AN: ED380483 
AU: Jones,-Russell-W. 

Tl: Performance and Alternative Assessment 
Techniques: Meeting the Challenge of Alternative 
Evaluation Strategies. 

PY: 1994 

NT: 29 p.; Paper presented at the International Conference 
on Educational Evaluation and Assessment (2nd, Pretoria, 
Republic of South Africa, July 1994). 

PR: EDRS Price - MF01/PC02 Plus Postage. 

DER: Educational-Trends; Evaluation-Methods; 
Knowledge-Level; Portfolios-Background-Materials; 

Test- Format 

DEM: •Educational- Assessment; •Multiple-Choice-Tests; 
•Portfolio-Assessment; •Test-Construction; Testing- 
AB: One of the most influential contemporary trends in 
educational evaluation in the United States is the move 
away from traditional testing methods toward "authentic 
assessments," which are designed to measure student 
performance of skills, abilities, and knowledge directly. 

While there is no consensus as to precisely what constitutes 
authentic assessment, there is general agreement that it 
incorporates: (1) emphasis on examinee performance, 
assessing not only what the examinee knows, but what the 
examinee can do; (2) use of direct methods of assessment; 
(3) inclusion of a high degree of realism; and (4) activities for 
which there may be no one correct answer, in a simulation 
of realism. The primary distinction between traditional 
testing methods and authentic assessment is the choice of 
question format. Certain segments of the educational 
community have called for a move from the multiple-choice 
format to question formats considered to reflect higher-order 
cognitive processes more accurately. This has resulted In 
the development or adoption of a broad range of formats, 
including standardized patient, audio-visual context setting, 
computer-based problem solving, multiple choice with 
justification, latent image, performance, and portfolio. Two 
figures illustrate the discussion. (Contains 30 references.) 
(Author/SLD) 



AN: ED378187 

AU: Foucar-Szocki, -Diane 

Tl: Becoming Assessors: Authentic Assessment for 
Authentic Instruction. A Report of the Blue Ridge 
Assessment Project, a Collaborative Effort of the 
Albemarle, Fluvanna, Greene, Harrisonburg, Orange, 
and Rockingham Schools. 

CS: Albemarle County Schools, Charlottesville, Va. 

PY: 1994 

AV: Frank Morgan, Curriculum Development and Research, 
Albemarle County Schools, 401 Mclntire Rd., Charlottesville, 
VA 22901. 

NT: 435 p.; Portions of section 4 (resources) contain illegible 
print. 

PR: EDRS Price - MF01/PC18 Plus Postage. 

DER: Check-Lists; Elementary-Education; 
Inservice-Teacher-Education; Performance-Tests; 
Simulation-; Skill-Analysis; Student-Projects 
DEM: "Evaluation-Methods; "Portfolio-Assessment; 
"Student-Evaluation; "Test-Construction 
AB: This report describes educators' collaborative 
self-directed experiences in learning about and producing 
alternative forms of assessment for use in elementary-level 
classrooms. The report analyzes the Virginia Standards of 
Learning and local curricula in Engitsh/Language Arts, 
Mathematics, Social Studies, and Science/Health in grades 
2-4, to provide common ground for the assessments. The 
assessments themselves are then presented, including 
checklists, portfolios, performance tasks, product 
assessments, projects, and simulations. Assessments in 
language arts cover oral communication, research and 
reporting skills, reading, writing, letter writing, and creative 
writing. Assessments in mathematics focus on problem 
solving, probability and statistics, data analysis, 
conservation, patterns, economics, money, measurement, 
geometry, and graphing. Social studies assessments 
address economics, earth care, Powhatan Indians, 

Tidewater region, and cooperative learning. Science/health 
assessments examine investigative skills and observation 
skills. A section titled "Voices" seasons the report with 
teachers’ statements about learning and individual change in 
their professional contexts and their lives. Another section 
offers recommendations to enhance teacher learning and 
change. A resources section presents a glossary, and Items 
for use as overhead transparencies or handouts. 

Appendices include a project plan, assessment item 
checklist, a model of the purposes of assessment, and a 
participant list. Contains 55 references. (JDD) 

AN: EJ492908 
AU: Hoerr,-Thomas-R. 

Tt: How the New City School Applies the Multiple 
Intelligences. 

PY: 1994 

JN: Educational-Leadership; v52 n3 p29-33 Nov 1994 
AV: UMI 

DER: Elementary-Education 
DEM: "Curriculum-Development; "Intelligence-; 
•Portfolios-Background-Materials; 
"Professional-Development; "Theory-Practice-Relationship 
AB: Describes a Saint Louis elementary school’s successful 
application of Howard Gardner's multiple intelligences 
theory. What began as a discussion of the nature of 
intelligence has resulted in a revised curriculum, varied 
instructional techniques, alternative assessment (using a 
combination of portfolios, progress reports, profiles, 
demonstrations of understanding, and standardized tests), 
improved professional development for teachers, and new 
ways to communicate with parents. (MLH) 



204 



Hot Topic Guide 59: Bibliography 4 



AN: EJ488411 

AU: Chariesworth, -Rosalind; And-Others 
Ti: Research on the Effects of Group Standardized 
Testing on Instruction, Pupils, and Teachers: New 
Directions for Policy. 

PY: 1994 

JN: Early-Education-and-Development; v5 n3 pi 95-21 2 Jul 
1994 

DER: Evaluation-Methods; Student-Evaluation; 
Test-Coaching; Test- Wiseness 
DEM: ‘Curriculum-Problems; ‘Group-Testing; 
*Standardized-Tests; ‘Test- Results; ‘Test-Use 
AB: Since the late 1 960s, curriculum and instruction have 
increasingly become ‘test driven/ Research documents the 
negative effects of standardized testing on instruction, 
students, and teachers. At their best, tests reflect students’ 
ability to take tests. Research supports the need for change 
in testing policies. Massive testing must be eliminated, and 
more authentic and performance-based evaluation should 
be Instituted. (TM) 

AN: ED374433 
AU: Calfee,-Robert-C. 

Ti: Ahead to the Past: Assessing Student Achievement 
In Writing. Occasional Paper No. 39. 

CS: National Center for the Study of Writing and Literacy, 
Berkeley, CA. 

PY: 1994 
NT: 14 p. 

PR: EDRS Price - MF01/PC01 Plus Postage. 

DER: Elementary-Secondary-Education; Higher-Education; 

Parent-School-Relationship 

DEM: *Portfolios-Background-Materials; 

‘Student-Evaluation; ‘Writing-Achievement; 

‘Writing-Evaluation 

AB: in the mid-1980s, partly through new developments in 
curriculum and instruction, a new assessment option has 
arisen under the banner of alternative assessment. The 
basic Idea appears In several guises: ‘authentic 
assessment* (implying that standardized tests are not 
authentic), "performance tests," and "portfolios." The goal of 
writing portfolios is to provide an opportunity for a richer, 
more authentic assessment of their achievements, to show 
their potential given adequate time and resources. A survey 
of a broad array of portfolio practices around the country 
finds that: (1) the portfolio approach is energizing the 
professional standing of classroom teachers; (2) 
respondents showed a distaste for evaluation; and (3) 
teachers had little concern about technical matters tike 
validity and reliability. Individual teachers interpret the 
portfolio concept quite differently in different settings. If 
portfolios are taken seriously, most students react seriously. 
One way to connect parents and schools is to place the 
students in a central role through the portfolio. Barriers to 
alternative assessments are substantial: time, money, 
motivation, and institutional support. The greatest hope for 
realizing the promise of portfolios may spring from the local 
school and the classroom teacher. Two caveats need to be 
observed: assessment practice and policies should be 
consistent for all teachers in a given school; and the 
audience and purpose for the assessment need to be 
established. Educators have made great strides during the 
past 50 years-the portfolio concept is but one example. 
(RS) 



AN: ED373082 

AU: Estrin,-Elise-Trumbull 

TI: Alternative Assessment: Issues in Language, 

Culture, and Equity. Knowledge Brief Number 11. 

CS: Far West Lab. for Educational Research and 
Development, San Francisco, Calif. 

PY: 1993 
NT: 9 p. 

PR: EDRS Price - MF01/PC01 Plus Postage. 

DER: Context- Effect; Elementary-Secondary-Education; 

Mi nority-G roups; Norm- Referenced -Tests; Performance-; 
Portfolios-Background-Materials; Social-Influences; 
Teaching-Methods; Test-Construction; Test- Validity; 

T rack-System- Education 

DEM: ‘Cultural-Differences; ‘Educational-Assessment; 
‘Equal- Education; "Language-Proficiency; "Test-Use 
AB: Alternative assessments, also called authentic or 
performance assessments, have first the common notion of 
a meaningful performance or product. The use of such 
alternatives with students belonging to nondominant 
language and cultural groups is of interest because of the 
hope that this type of assessment can show what these 
students can really do and because of the fear that the 
inequities that are associated with traditional 
norm-referenced tests will recur. Negative consequences of 
traditional forms of assessment for students from 
nondominant cultural and language groups are well 
documented. Equity issues in the public debate on any 
assessment are consequential validity, gatekeeping, 
tracking, and the opportunity to learn. The social context of 
assessment must be acknowledged in the design of 
alternative assessments. The alternative-assessment 
approach is compatible with a constructivist view of learners. 
Some suggestions are offered for a culturally responsive 
pedagogy and assessment design. A particular focus is on 
portfolio use in the pluralistic classroom. Three tables and 
one figure illustrate the discussion. (Contains 22 
references.) (SLD) 
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